Search docs.molo17.com
    Operational guide Deploy and run

    Upgrade Gluesync safely

    This guide shows how to move a Gluesync installation on Linux to a newer version without losing configuration or changes. You write down what runs now, back up the data folder, pick one of three ways to upgrade (the Control Plane, Gluesync Control or an offline package), check that your pipelines resume, and know how to go back if something is wrong.

    In this guide

    You will learn what to check before you upgrade, how to back up the data folder, whether to pause your pipelines, how the three upgrade paths differ, how to confirm that the new version runs and that change data capture (CDC) continues, which of your settings an upgrade keeps, and how to roll back with Gluesync Control and your backup.

    Quick information

    Goal

    Upgrade Core Hub and the platform modules and be able to prove that nothing was lost

    Audience

    Gluesync administrators with sudo access to the Linux server

    Level

    Intermediate

    Time required

    About 45 minutes, including the backup and the checks. The upgrade itself took 3 to 20 minutes in the lab, depending on the path

    Area

    Control Plane → Platform updates; Gluesync Control (gluesync update)

    Applies to

    Gluesync 2.3

    Before you begin

    Make sure that:

    1. Gluesync was installed with the Linux installer, so that gluesync is on the PATH (check with gluesync --version). See Install Gluesync on Linux and size the host;

    2. You can use sudo on the server and sign in to the Control Plane as a user who may update the platform;

    3. The disk has room for the new images next to the old ones: gluesync update --dry-run prints the rough amount (4.7 GB for a full update, 6.3 GB for the offline package of the lab);

    4. You have read the release notes of the version you are going to, in Release notes, especially any upgrade note at the top;

    5. You have a maintenance window: Core Hub restarts during the upgrade, and the Control Plane is unavailable for a short time. Use a test system first if you can, as the 2.3 release is large.

    Core Hub migrates its database the first time it starts with a newer version, and that migration is one-way. Gluesync copies the database before it migrates it (gluesync.db.pre-v<old version>.bak in the database folder), but only your own backup of the whole data folder brings you back to the exact state before the upgrade.

    This guide was tested on Rocky Linux 9.8 with Gluesync 2.3.0.1 going to 2.3.1.1, on a pipeline from PostgreSQL to PostgreSQL that received one change per second during every upgrade, and on a server that went from 2.2.11.12 to 2.3.1.1. Your version numbers will differ.

    Guide steps

    Write down what runs now

    CLI

    1. On the server, print the state of Gluesync and the versions it can move to:

      sudo gluesync status
sudo gluesync update --list
sudo gluesync update --services
sudo gluesync update --pins

    2. In the Control Plane, open Settings → About and note the version shown;

    3. Copy the output of the commands and the values of the settings you changed (Settings → Global settings and Support) into your change record.

    Expected result

    gluesync status reports Running and 7 of 7 running. gluesync update --list shows the installed version, the channel and the versions offered. --services lists every service with its installed and latest version, and --pins shows whether a version is pinned (a pin stops updates of that service, see the last step).

    The Control Plane shows the version under the logo: 2.3.0 - build 1 is version 2.3.0.1, and 2.3.1 - build 1 is 2.3.1.1.
    Check the plan and the free disk space

    CLI

    1. Check the free space where Docker keeps its images and where Gluesync lives:

      df -h /opt/gluesync /var/lib/docker

    2. Ask Gluesync Control what it would do. Nothing changes:

      sudo gluesync update --dry-run
    Expected result

    The dry run lists every image that would change (seven for a full update: Core Hub, Chronos, Conductor, the reverse proxy, Prometheus, Grafana and Portainer), the rough disk space it needs, and ends with Dry run: 7 image(s) would change; nothing was changed. It also prints the warning that the database migration of Core Hub is one-way.

    Back up the data folder

    CLI

    A copy of the whole installation folder is the backup that lets you go back. It holds the compose file, .env and the data of Core Hub, which has its databases in data/core-hub/database. Copy it while Gluesync is stopped, so that the databases are not changing while you copy.

    1. Stop Gluesync. Your data and your images stay:

      sudo gluesync stop

    2. Make the archive, and keep a copy off this server:

      sudo mkdir -p /var/backups/gluesync
sudo tar czf /var/backups/gluesync/gluesync-$(date +%Y%m%d)-pre-upgrade.tar.gz -C /opt gluesync
ls -lh /var/backups/gluesync

    3. Start Gluesync again:

      sudo gluesync start
    Expected result

    The archive exists and is not empty (5.3 MB in the lab, where the installation folder was 15 MB, and the copy took about one second). Gluesync runs again after about 10 seconds, and pipelines that were active resume by themselves.

    You can also keep a configuration backup of your pipelines, agents and user-defined functions with Gluesync Automator, see Platform configuration backup and restore, or an encrypted backup with Gluesync Connect, see Backups with Gluesync Connect. They do not replace the copy above: they hold configuration, not the exact state of the databases.

    The source and target databases are not part of Gluesync and are not in this backup.
    Decide whether to pause the pipelines

    UI

    You do not have to pause pipelines for an upgrade. In the lab, a source table received one insert per second during every upgrade, while Core Hub restarted and the platform updated. When Core Hub was back, the pipeline was active again, and the target held the same number of rows as the source, with the last inserted row present. Core Hub continues from the position it stored.

    Still, wait before you start in these cases:

    • An entity is running its initial snapshot: let it finish (this was not tested);

    • You have a large backlog on a source agent and little time: the restart adds to the time the backlog needs to drain.

    If you want to pause anyway, use Pause Section on the pipeline in the Control Plane (see Play and pause entities) and resume the entities after the upgrade.

    Choose how to upgrade

    UI + CLI

    The three paths end in the same place: the images of the services in docker-compose.yml have new tags and the containers were recreated. They differ in how you drive them and in what they update.

    Path Where you run it What it updates What to expect in the lab

    Control Plane: Platform updates

    Browser, no shell. Needs internet access from the server

    Every service in the list: Conductor, Chronos, the reverse proxy, Grafana, Portainer, Prometheus and Core Hub, one at a time

    About 20 minutes with image pulls. The window said "Update failed" for the Conductor and for the reverse proxy although both were updated, and the whole list needed three clicks of Update all. Comments in docker-compose.yml were removed

    Gluesync Control: gluesync update

    Terminal, with sudo. Needs internet access from the server

    All seven images by default, or one service or one pin with options

    About 2 minutes 40 seconds. Prints the plan first, makes a backup of docker-compose.yml and .env, checks the result and rolls back by itself if the services do not come up. Comments in docker-compose.yml are kept

    Offline package: gluesync update --package

    Terminal, with sudo. No internet access needed

    Only what the package carries: Core Hub and Chronos

    About 4 minutes for a 3.2 GB package, which is read twice. Comments in docker-compose.yml are kept

    Use the Control Plane when nobody can use a shell and the server is online. Use Gluesync Control when you can, because it shows the plan, keeps a backup of the compose file, rolls back a failed update and can be scripted. Use the offline package when the server cannot reach the internet.

    Upgrade with Gluesync Control

    CLI

    1. Update to the newest version of your release channel. Add --version <x.y.z.w> to choose another one, or --service <name> to update a single service:

      sudo gluesync update

    2. Answer Yes when it asks for confirmation (the installed version, the new one and the warning about the one-way migration). Add -y in a script;

    3. Wait for the five steps to finish: checking the installation, backing up the configuration, downloading the new images, restarting the services and checking that they run.

    You can do the same in the terminal screen: run sudo gluesync with no arguments, press 2 for the Update tab, choose Check for updates, then select a version to see the plan and confirm it.

    Gluesync Control terminal screen on the Update tab showing the plan to update from 2.3.0.1 to 2.3.1.1 with seven image changes
    Figure 1. The plan in the terminal screen lists every image change and the one-way migration warning before anything is changed.
    Expected result

    The command ends with Gluesync updated from <old> to <new>., prints where it kept the previous docker-compose.yml and .env (a folder under /var/lib/gluesync/backups), and exits with 0.

    If the update cannot complete, Gluesync Control puts the previous images and files back and says so. It does not roll back the database, see the last step.

    Or upgrade from the Control Plane

    UI

    1. Sign in to the Control Plane. When updates are available, Platform updates shows a counter next to the notification icon. Select it;

    2. In the alert, select View details;

    3. Open an item to read its changelog, then select Update all;

    4. Leave the window open and do not refresh the page until every item has a green check. Core Hub signs you out when it restarts;

    5. If an item shows Update failed, open Show error details, check with docker ps on the server which version runs, then select Retry or Update all again.

    Top bar of the Control Plane with the Platform updates button marked by annotation 1
    Figure 2. The counter on Platform updates tells you how many services have a newer version.
    Platform updates alert with the View details button marked by annotation 2
    Figure 3. Select View details to open the list of updates.
    Platform updates available window listing seven services with Update all marked by annotation 3
    Figure 4. The modal lists every service with its version change. Select Update all.
    Expected result

    The services update one after the other, and the items get a green check. In the lab it took three runs of Update all, because the item that updated the Conductor ended with the error below even though the Conductor was running its new version, and the reverse proxy item ended with Request failed with status code 404 while the proxy was already on its new version. Each run continued with the services that were left, and the list got shorter by the services that were done. After the last one, Core Hub restarted and gluesync status showed the new version.

    Platform updates window with the Gluesync Conductor item marked as failed
    Figure 5. The Conductor and the reverse proxy restart themselves during the update, so their item can show an error although the update was applied.
    Or upgrade offline

    CLI

    1. On a computer with internet access, download gluesync-update-linux-<amd64|arm64>-<version>.tar.gz for the CPU of the server from Get Gluesync - offline downloads. Copy it to the server;

    2. Check what it would do. This reads the whole package once:

      sudo gluesync update --package ~/offline/gluesync-update-linux-amd64-2.3.1.1.tar.gz --dry-run

    3. Run the update:

      sudo gluesync update --package ~/offline/gluesync-update-linux-amd64-2.3.1.1.tar.gz -y
    Expected result

    The command ends with Gluesync updated from <old> to <new>., as an online update does, after reading the package, backing up the configuration, loading the images from it (done (2 images) in the lab), restarting and checking the services. It took 3 minutes 52 seconds for a 3.2 GB package. Only Core Hub and Chronos changed: Conductor, the reverse proxy, Prometheus, Grafana and Portainer stayed on the versions they had, because this package does not carry them. Update those from an online path, or with their own packages, when you can.

    Check the result

    CLI + UI

    1. Check the version and the services:

      sudo gluesync status
docker ps --format '{{.Names}} {{.Image}}'

    2. In the Control Plane, open Settings → About and compare the version with the one you wrote down;

    3. Open Data pipelines and check that your entities are active and that new changes arrive: change a row in the source and look at the target, or compare the counts of the source and target tables;

    4. Check that the settings you noted are unchanged (Settings → Global settings and Support);

    5. Compare the compose file with the copy that Gluesync Control kept, which is in the folder it printed:

      sudo diff /var/lib/gluesync/backups/<date-time>/docker-compose.yml /opt/gluesync/docker-compose.yml
    Control Plane top bar after the upgrade with the version 2.3.1 build 1 under the Gluesync logo marked by a red box
    Figure 6. The version of the new installation is shown under the Gluesync logo.
    Expected result

    gluesync status shows the new version and 7 of 7 running. The pipeline is active, and the source and target tables hold the same rows, including the rows inserted while Gluesync was updating. The diff shows only the new image tags.

    What an upgrade keeps

    Customers sometimes report that settings were cleared after an upgrade. In the lab these were set before the upgrade and read back afterwards, through the Control Plane API and the files of the installation:

    What Control Plane update gluesync update Offline package

    Support → log retention (Max history (Days), Total size cap (GB)) and log level

    Kept

    Kept

    Kept

    Global settings → Grafana URL and release channel

    Kept

    Kept

    Kept

    Your own lines in docker-compose.yml (an environment variable, mem_limit, the Prometheus retention flag)

    Kept

    Kept

    Kept

    Comments in docker-compose.yml

    Removed

    Kept

    Kept

    .env (your own variables)

    Kept

    Kept

    Kept

    Pipelines, entities, connections

    Kept

    Kept

    Kept

    The same values were also kept when a server went from 2.2.11.12 straight to 2.3.1.1 with gluesync update (log retention, log level, Grafana URL, your lines and comments in docker-compose.yml, and .env). No pipeline was running on that server, so the pipeline row of the table was not tested for the 2.2 to 2.3 step.

    Two things can still look like a lost setting:

    • The Control Plane update rewrites docker-compose.yml. Your changes stay, but the comments do not, so keep your own notes outside the file. A change that you make by editing a service block is only applied when Gluesync recreates the container;

    • A pin of a service survives updates and stops that service from moving. If a service did not update, run sudo gluesync update --pins, and sudo gluesync update --unpin <service> or --unpin all when the pin is no longer wanted.

    Gluesync Control writes the previous docker-compose.yml and .env to /var/lib/gluesync/backups/<date-time> on every update and keeps the last five, so you can always compare.

    Go back to the previous version

    CLI

    Do this only if the new version does not work for you. Rolling back needs the data backup of the earlier step. Going back to an older image is not enough: the older Core Hub cannot read a database that the newer one migrated. So you first put the older image back, then you put the old data back.

    1. Pin Core Hub to the previous version. --allow-downgrade is required, and the pin copies Core Hub’s data folder to a backup folder first:

      sudo gluesync update --pin core-hub=<previous version> --i-understand-pinning --allow-downgrade -y

      Expect the command to report success and gluesync status to show the older version, while the Control Plane still answers 502 Bad Gateway. Core Hub is running the older image on the migrated database and cannot open it;

    2. Stop Gluesync:

      sudo gluesync stop

    3. Put the data of Core Hub from your backup back. Move the current folder aside rather than deleting it:

      sudo mv /opt/gluesync/data/core-hub /opt/gluesync/data/core-hub.after-upgrade
sudo tar xzf /var/backups/gluesync/gluesync-<date>-pre-upgrade.tar.gz -C /opt gluesync/data/core-hub

    4. Start Gluesync:

      sudo gluesync start

    5. Check the result as in the previous step.

    Expected result

    gluesync status shows the previous version, you can sign in to the Control Plane, the settings are the ones you had before the upgrade, and the pipeline is active again. In the lab the counts of the source and target tables were the same a few seconds after the start, although rows had been inserted while Gluesync was down.

    Changes you made in Gluesync after the backup are gone after a rollback, because the databases of Core Hub are the ones of the backup. The source and target databases are not touched.

    When you are ready to move forward again, remove the pin with sudo gluesync update --unpin core-hub, then upgrade as above.

    Verify the result

    You have upgraded Gluesync safely if:

    • You noted the versions, the settings and the pins before you started;

    • A copy of the installation folder made with Gluesync stopped exists, and a copy is off the server;

    • gluesync status shows the new version and 7 of 7 running, and Settings → About agrees;

    • Your pipelines are active and the source and target tables hold the same rows;

    • The settings and the lines you added to docker-compose.yml are still there;

    • You know the commands to go back, and where the backup is.

    Troubleshooting

    Problem Possible cause What to do

    Update failed on the Conductor or the reverse proxy in Platform updates, with Conductor is updating or Request failed with status code 404

    The service restarted itself while the Control Plane was waiting for its answer

    Check the version with docker ps, then select Retry or Update all again. Use gluesync update if it keeps happening

    A service does not move to the new version

    A pin of Gluesync Control

    sudo gluesync update --pins, then --unpin <service>

    After going back to an older Core Hub, the Control Plane answers 502 Bad Gateway and the log says A migration from <new> to <old> was required but not found

    Core Hub cannot open a database that a newer version migrated. gluesync status can still say Running

    Restore the data of Core Hub from your backup (the rollback step)

    Related guides

    Contact MOLO17 Support if you need assistance with upgrading Gluesync.