Search docs.molo17.com
    Operational guide Deploy and run

    Deploy and update Gluesync on Docker Swarm

    This guide takes two Linux servers from "Docker installed" to a Gluesync stack running on Docker Swarm. You create the swarm, prepare the node that keeps the data, deploy the Swarm kit from the manager and sign in. Then you update a service from the Control Plane, with Swarm rolling it back if it fails, and take a backup you can restore.

    In this guide

    You will learn which node runs which service, how to create the swarm and label the data node, how to deploy the gluesync stack with the kit’s prepare-host.sh and deploy.sh, how to check the services from the command line and from the Control Plane, how Gluesync Conductor updates a Swarm service and writes the new version back to docker-stack.yml, how to change the stack safely, and how to back up the data node.

    Quick information

    Goal

    Run Gluesync as a Docker Swarm stack, and keep it up to date from the Control Plane

    Audience

    System administrators who already run Docker Swarm, or who must deploy Gluesync on it

    Level

    Intermediate

    Time required

    About 30 minutes, plus the download time of the images on the nodes

    Area

    Deploy and run → Docker Swarm (gluesync-docker-swarm directory of the kit); Control Plane → Platform updates

    Applies to

    Gluesync 2.3

    Before you begin

    Make sure that:

    1. You have one or more Linux servers with Docker Engine 23 or later, and an account that can use sudo and the docker command on each of them;

    2. You have your Gluesync kit, <kit ID>-trial-kit.zip, from the link in your Gluesync download email or from Get Gluesync, and it contains a gluesync-docker-swarm directory;

    3. The servers reach each other on the Swarm ports: 2377/tcp to the managers, 7946/tcp and 7946/udp, and 4789/udp. See Docker’s Swarm ports;

    4. Ports 80 and 443 are free on the manager, or you have chosen other ports (see Change the stack and deploy again);

    5. The nodes can pull images from Docker Hub.

    The commands in this guide were run on a two-node swarm with Docker Engine 29.9 and Linux containers: one manager (gs-manager) and one worker that holds the data (gs-data). The images come from Docker Hub for both amd64 and arm64. Replace the node names and addresses with your own. To run Gluesync on Windows Server 2022 instead, see On Windows Server 2022.

    Plan the nodes

    The Linux Swarm kit uses two node roles. One machine can have both, so a single-node swarm works too:

    Role How the kit finds it What runs there

    Manager

    A Swarm manager. deploy.sh labels it gluesync.kit=true

    Traefik (reverse-proxy) and Gluesync Conductor. The kit directory lives here, because Conductor redeploys the stack from it

    Data node

    The node labelled gluesync.data=true. prepare-host.sh creates its data folders

    Core Hub, Chronos, Prometheus and Grafana. All persistent data is in host folders under GLUESYNC_DATA_ROOT (default /var/lib/gluesync-swarm)

    The stateful services are pinned to the data node because a Swarm named volume is local to one node: a task moved to another node would start with an empty volume. Core Hub runs as one replica. It has no clustering, so adding replicas does not make it highly available.

    Conductor must run on a manager, because only managers answer the Swarm API for services, tasks and service logs.

    Guide steps

    Create the swarm and join the data node

    CLI

    1. On the server that will be the manager, create the swarm:

      docker swarm init

      On a server with more than one network address, add --advertise-addr <address> with the address the other nodes reach it on.

    2. Print the command that joins a worker:

      docker swarm join-token worker

    3. Run the printed docker swarm join --token <token> <manager address>:2377 command on the data node;

    4. On the manager, list the nodes:

      docker node ls
    Expected result

    The data node prints This node joined a swarm as a worker., and docker node ls lists both nodes as Ready, with Leader next to the manager:

    ID                            HOSTNAME     STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
vcd9xgo40qsi0b8yzac2gwchd     gs-data      Ready     Active                          29.9.2
kwn9yqsgif418jms1rnfm5jfv *   gs-manager   Ready     Active         Leader           29.9.2

    To use one server for everything, run only docker swarm init on it and skip the join.

    Prepare the data node

    CLI

    prepare-host.sh creates the data folders and copies into them the licence, the certificates and the configuration of your kit. Run it on the data node.

    1. Unzip the kit, and copy its gluesync-docker-swarm directory to the data node;

    2. In that directory, run the script as root:

      sudo ./prepare-host.sh

    3. If the data node is a worker, the script cannot label it. Run the label command it prints on the manager, with the hostname or the ID of the data node:

      docker node update --label-add gluesync.data=true gs-data

    To keep the data somewhere else than /var/lib/gluesync-swarm, set GLUESYNC_DATA_ROOT in .env before you run the script on the data node, and use the same value in the .env of the manager.

    Expected result

    The script lists each file it copies and ends with the next step:

    Preparing host data root: /var/lib/gluesync-swarm
  seeded /var/lib/gluesync-swarm/shared/gluesync-cert.pem
  seeded /var/lib/gluesync-swarm/shared/gluesync-key.pem
  seeded /var/lib/gluesync-swarm/shared/gluesync.com.jks
  seeded /var/lib/gluesync-swarm/shared/gs-license.dat
  seeded /var/lib/gluesync-swarm/shared/logback.xml
  seeded /var/lib/gluesync-swarm/shared/security-config.json
  seeded /var/lib/gluesync-swarm/data/core-hub/bootstrap-core-hub.json
This node is not a swarm manager. On a manager, label it:
  docker node update --label-add gluesync.data=true <node-id>
Done. Next, run ./deploy.sh on the manager that holds the kit.

    On a data node that is also the manager, the label is added for you and the two This node is not a swarm manager lines are missing.

    The script copies the kit files over the files already in GLUESYNC_DATA_ROOT/shared. Run it once, on a new data node. To replace only the licence later, see Troubleshooting.
    Deploy the stack from the manager

    CLI

    1. Copy the same gluesync-docker-swarm directory to the manager, to the folder where it will stay. Conductor mounts this folder and redeploys the stack from it, so do not move it afterwards;

    2. In that directory, deploy the stack:

      ./deploy.sh

      The script creates .env from .env.example, writes the absolute path of the directory into it, labels the manager gluesync.kit=true and runs docker stack deploy for a stack named gluesync.

    3. Watch the services start while the nodes pull the images:

      docker stack services gluesync

    4. Check where each task runs:

      docker stack ps gluesync --filter desired-state=running --format "{{.Name}}\t{{.Node}}\t{{.CurrentState}}"
    Expected result

    deploy.sh creates the network gluesync-net, two secrets, three configs and six services, and ends with Stack deployed. After a few minutes every service shows 1/1, and the tasks run on the nodes of Plan the nodes:

    gluesync_gluesync-chronos.1     gs-data      Running 3 minutes ago
gluesync_gluesync-conductor.1   gs-manager   Running 3 minutes ago
gluesync_gluesync-core-hub.1    gs-data      Running 3 minutes ago
gluesync_grafana.1              gs-data      Running 3 minutes ago
gluesync_prometheus.1           gs-data      Running 3 minutes ago
gluesync_reverse-proxy.1        gs-manager   Running 3 minutes ago

    docker stack services gluesync shows the image of each service. The kit writes literal versions, such as molo17/gluesync-chronos:<version>, never latest.

    If gluesync_gluesync-conductor is missing from the list, your kit was packaged when the available Conductor did not support Swarm (Swarm mode needs Conductor 0.14.0 or later). The stack works, but Platform updates cannot update it. Download a new kit from Get Gluesync, or update the services by hand as described in Change the stack and deploy again.
    Sign in and check the services

    UI

    1. Open https://<manager address>/. Port 80 redirects to 443. The kit certificate is self-signed, so the browser shows a warning the first time;

    2. Sign in with admin / admin, then create your own password when Gluesync asks for it;

    3. In the left menu, select Modules Manager.

    Modules Manager with six healthy rows: Gluesync CoreHub
    Figure 1. Modules Manager lists the six services of the stack. The nickname is the name of the Swarm task, and the version comes from the image of the service.
    Expected result

    All six services are Healthy. Grafana answers at /grafana (admin / password, set in .env) and Prometheus at /prometheus. The Control Plane, Modules Manager and Platform updates work as they do on Docker Compose: Conductor recognizes the Swarm stack from the labels of its own service and works on the Swarm services instead of containers.

    The kit does not deploy agents. Add them from Core Hub when the platform is running: Conductor adds each agent to docker-stack.yml and deploys it. Then continue with Build your first pipeline.

    Update services from Platform updates

    UI

    When a newer version of a service is published, the Platform updates icon (a cloud with a down arrow) in the top bar shows a badge.

    1. Select Platform updates, then View details;

    2. Expand a service to read what changed, then select Update all.

    Platform updates available dialog listing Gluesync Chronos with the from and to versions and the release date
    Figure 2. Platform updates on the Swarm stack of the lab: Chronos had been deployed one version back, so Conductor offered the newer one with its release notes.

    For each service, Conductor sets the new image with automatic rollback as the failure action, waits until all replicas have kept running for 20 seconds (UPDATE_VERIFY_SECONDS), and then writes the new tag to docker-stack.yml in the kit directory. Swarm pulls the image while it starts the new task, and the download time does not count against the update timeout.

    Update complete dialog saying that all components are up to date and listing the Chronos update that was applied
    Figure 3. The update of Chronos finished in about a minute and a half in the lab.
    Expected result

    The dialog ends with Update complete. On the manager, the stack file and the service show the new version:

    grep -n 'gluesync-chronos:' docker-stack.yml
docker service inspect gluesync_gluesync-chronos --format '{{.Spec.TaskTemplate.ContainerSpec.Image}}'

    The first command prints the new tag. The second prints the same tag followed by @sha256: and the digest of the image: Conductor pins the service to that image, so running ./deploy.sh again later does not restart it. In the lab, a second ./deploy.sh left the updated Chronos task running.

    If a new version does not stay up, Swarm rolls the service back to its previous image, or Conductor does it when Swarm has not. The update is reported as failed, and docker-stack.yml keeps the previous tag. When an update of Conductor itself is offered, Conductor answers first and then updates its own service. In Swarm mode it never updates itself to an older version, so a release without Swarm support is not installed.

    For the full list of what Conductor does in Swarm mode, and the settings that tune it, see Conductor in Swarm mode.

    Change the stack and deploy again

    Config

    docker-stack.yml and .env in the kit directory on the manager are the source of truth of the stack. To change it, edit them there and deploy again:

    1. Edit the file:

      • To publish other ports, set TRAEFIK_HTTP_PORT and TRAEFIK_HTTPS_PORT in .env;

      • To give services CPU or memory limits, add deploy.resources to them in docker-stack.yml. The kit sets none, because Swarm rejects a limit above the number of CPUs of a node;

      • To change a version by hand, edit the image: line of the service. Keep it a literal repo:tag value: Conductor rewrites that line when it updates the service, and leaves alone an image written with a variable such as ${VAR};

    2. Deploy again from the kit directory:

      ./deploy.sh
    Expected result

    docker stack deploy prints Updating service for each service, and only the services whose definition changed get new tasks. docker stack deploy does not read .env by itself: deploy.sh and Conductor load it before they deploy, so always use deploy.sh.

    docker stack rm gluesync removes every service of the stack. The data in GLUESYNC_DATA_ROOT stays, but Gluesync is down until you run ./deploy.sh again.
    Back up the stack

    CLI

    A backup of a Swarm stack has two parts: the data folders on the data node, and the kit directory on the manager.

    1. On the manager, stop the services that write data:

      docker service scale --detach=false gluesync_gluesync-core-hub=0 gluesync_gluesync-chronos=0 gluesync_prometheus=0

    2. On the data node, archive the data root:

      sudo tar -czf /root/gluesync-data-$(date +%Y%m%d-%H%M).tar.gz -C /var/lib gluesync-swarm

    3. On the manager, start the services again:

      docker service scale --detach=false gluesync_gluesync-core-hub=1 gluesync_gluesync-chronos=1 gluesync_prometheus=1

    4. Also copy the kit directory of the manager, with its .env, docker-stack.yml and shared folder, whenever you change it or after an update.

    Expected result

    Each scale command ends with verify: Service gluesync_<name> converged. In the lab the whole backup took about 40 seconds, the services came back 1/1, and the password changed at the first sign-in still worked. Copy the archive off the data node.

    To restore, deploy the kit on the new nodes as in this guide, scale the same three services to 0, replace GLUESYNC_DATA_ROOT on the data node with the content of the archive (sudo tar -xzf <archive> -C /var/lib), and scale them back to 1. In the lab, a data root that was deleted and restored this way came back with its users and settings. For what Core Hub keeps in its data folder, and for backups without downtime, see Back up, restore and move Core Hub and Backup and restore.

    On Windows Server 2022

    The kit also has a gluesync-docker-swarm-windows directory, for a single-node swarm on Windows Server 2022 with the latest cumulative update and Windows containers. Every service runs on that one node, and the data stays in the data and shared folders of the kit directory. In PowerShell as Administrator, run .\prepare-host.ps1, which also creates the swarm and opens the firewall, then .\deploy.ps1. The requirements, such as MAC address spoofing on a Hyper-V virtual machine, are in Install on Windows Server 2022.

    Troubleshooting

    Problem Possible cause What to do

    deploy.sh stops with error: this node is not a swarm manager

    The script runs on a worker, or on a server outside the swarm

    Run it on a manager, in the kit directory that will stay there, or run docker swarm init first

    deploy.sh stops with error: no node is labeled gluesync.data=true

    The data node was prepared on a worker and the label command was not run

    On a manager, run docker node update --label-add gluesync.data=true <data node>

    Core Hub, Chronos, Prometheus and Grafana stay 0/1, and docker service ps shows no suitable node

    No node has the gluesync.data=true label, or the labelled node is down

    Check the labels with docker node inspect <node> --format '{{json .Spec.Labels}}' and the node state with docker node ls

    Core Hub starts, but Conductor restarts again and again, its log shows License not valid or expired, and Chronos is restarted as unhealthy

    The licence in the data folder, or in the kit shared folder that Conductor reads on the manager, is missing, expired or not valid

    Copy the new gs-license.dat to GLUESYNC_DATA_ROOT/shared on the data node and to the kit shared folder on the manager, then run docker service update --force on gluesync_gluesync-core-hub, gluesync_gluesync-chronos and gluesync_gluesync-conductor

    Conductor answers every request with SWARM_NOT_MANAGER

    The Conductor task runs on a worker

    Keep the placement of the kit (node.role == manager and node.labels.gluesync.kit == true) and run deploy.sh on the manager that holds the kit directory

    gluesync_gluesync-conductor is not in the stack

    The kit was packaged before Conductor supported Swarm

    Download a new kit, or update the services by editing docker-stack.yml and running ./deploy.sh

    A service is rejected after you add resource limits

    A CPU limit or reservation above the CPUs of the node

    Lower the value in deploy.resources to fit the node, then run ./deploy.sh

    The Control Plane page does not open

    Port 80 or 443 is already in use on the manager, or blocked by a firewall

    Set other ports in .env and run ./deploy.sh, or free the ports

    An update in Platform updates is reported as failed

    The new version did not stay up for the verification time, so it was rolled back

    Read the reason with docker service ps --no-trunc gluesync_<service> and docker service logs gluesync_<service>. The stack file keeps the previous version

    For the diagnosis codes Conductor returns, see Conductor error codes. For problems inside Gluesync, follow Read Gluesync logs and diagnose errors: on Swarm, read the logs with docker service logs gluesync_<service>, which includes the tasks on every node.

    Related guides

    Contact MOLO17 Support if you need assistance with running Gluesync on Docker Swarm.