Search docs.molo17.com

    Gluesync on Podman

    Gluesync runs on Podman as well as on Docker Engine, on Linux. Gluesync Control installs and manages it in two modes: rootless, where an ordinary user owns the containers, and rootful, where root does. The kit is the same one used for Docker. Only the engine, its socket and a few kit settings change.

    Docker stays the default everywhere. Nothing changes for an install made without --runtime podman, and the Windows installers and Windows containers keep using Docker. This page replaces the earlier manual setup with podman-compose: Gluesync is driven with Docker Compose v2 against the Podman socket, and podman-compose is not used.

    Rootless or rootful

    Rootless Rootful

    Who runs the containers

    The owner: the user who runs sudo gluesync install. Never root.

    root

    Engine socket (the Docker-compatible API that Compose talks to)

    /run/user/<uid>/podman/podman.sock

    /run/podman/podman.sock

    Chosen by default when

    sudo is run by an ordinary user

    The installer is started by root without sudo

    Files of the install and of Core Hub’s data

    Owned by the owner, so the owner can read the data and the logs

    Owned as on Docker

    Ports 80 and 443

    Need net.ipv4.ip_unprivileged_port_start=80, which the installer sets

    No change

    Extra setup

    Sub-user and sub-group IDs, lingering, the sysctl above

    None

    Pick rootless when the security policy of the server asks for containers that never run as root, or when Gluesync belongs to one person or team account. Pick rootful when you want the least moving parts, or when the server has no ordinary user to own the containers. A rootless install needs Core Hub 2.3.2.0 or later and Chronos 0.3.3 or later (see Requirements).

    Supported platforms

    • Linux only, on the distribution families of Gluesync Control for Linux: Ubuntu, Debian, the Red Hat family (RHEL, Rocky Linux, AlmaLinux, Oracle Linux, CentOS), Amazon Linux, and SUSE Linux Enterprise Server and openSUSE.

    • systemd is required: Podman is reached through its socket unit.

    • Tested: Ubuntu 24.04 with Podman 4.9.3 (rootless), and Rocky Linux 9.8 with SELinux enforcing and Podman 5.8.2 (rootful). On both: online install, start, restart, status and collect-logs, a Conductor restart, stop/start and update, migration of a kit generated before Podman support, and a reboot. Rootless only: offline install and the uninstaller.

    • Not supported on Podman: Docker Swarm (see Docker Swarm), Windows containers, and the Windows installers. Podman Desktop and podman machine on Windows or macOS are not a Gluesync deployment target.

    Requirements

    • Podman 4.9 or later. The installer installs it, online. Offline, it must already be installed.

    • Docker Compose v2, the docker-compose-plugin package from Docker’s repository (or docker-compose-v2 on Ubuntu). The installer installs it when it is missing, online. It installs only the plugin, never Docker Engine. podman-compose is not used and is not a substitute.

    • cgroup v2, the default on every supported distribution. stat -fc %T /sys/fs/cgroup prints cgroup2fs when it is on.

    • Rootless only:

      • an ordinary user with a home folder;

      • a range of at least 65536 sub-user and sub-group IDs for that user in /etc/subuid and /etc/subgid: the installer adds one when it is missing;

      • Core Hub 2.3.2.0 or later and Chronos 0.3.3 or later in the kit: a rootless Podman runs both as a non-root user (uid 1017) instead of as root, so the data of the install stays readable by the owner.

    • The resources and the free disk space of System requirements. On Podman the free space is checked on the graph root of podman info: the owner’s ~/.local/share/containers/storage for a rootless install.

    Rootless needs Core Hub 2.3.2.0 or later and Chronos 0.3.3 or later. Earlier images fail to start in a rootless install, because their entrypoint expects to start as root, and the containers restart forever. Rootful Podman, Docker and Docker Swarm run the containers as root and have no such requirement.

    gluesync install --runtime podman --rootless reads the Core Hub and Chronos images of the kit (or, offline, of the package) before it copies or starts anything. When one is older, it stops with GS-RUN-PODMAN_ROOTLESS_IMAGES (exit 100), see Installer codes. Install with --rootful or --runtime docker, or use a kit or package with Core Hub 2.3.2.0 or later and Chronos 0.3.3 or later. An image tag with no version, such as latest, is only a warning.

    Install with Gluesync Control

    Gluesync Control installs Podman, Compose and the Gluesync kit in one run, as on Docker. See Install Gluesync for everything that is not specific to Podman: the kit ID, the checks, the options and the exit codes.

    # Rootless: run it with sudo from the account that will own Gluesync
    sudo gluesync install --runtime podman --kit a1b2c3d4e5f6 --start
    
    # The same, written out
    sudo gluesync install --runtime podman --rootless --kit a1b2c3d4e5f6 --start
    
    # Rootful
    sudo gluesync install --runtime podman --rootful --kit a1b2c3d4e5f6 --start

    The two scripts of the kit pass their arguments on, so curl -fsSL https://molo17.com/gs-content/utils/docker-installer.sh | sudo bash -s — --runtime podman works too.

    Options and variables

    Option Variable Meaning

    --runtime <docker|podman>

    GLUESYNC_RUNTIME

    The container runtime. docker is the default. Any other value stops with GS-CTL-006 (exit 255).

    --rootless, --rootful

    GLUESYNC_PODMAN_MODE=rootless or rootful

    The Podman mode. Either option implies --runtime podman. With --runtime docker, or both together, they are an error. Without them, the mode is rootless when sudo was run by an ordinary user (SUDO_USER), and rootful otherwise.

    --rootless started by root without sudo (no ordinary user to own the containers) stops with GS-RUN-PODMAN_NO_USER (exit 90) before anything changes. Run sudo gluesync install --runtime podman from the account that will own Gluesync, or use --rootful.

    The terminal screen

    In a terminal, sudo gluesync install shows the engine on its Docker page. Press Up and Down to choose between Docker (default), Podman rootless and Podman rootful. The commands, the list of what is set up and the Ready page follow the choice. The screen starts on Podman rootless only when Podman is installed on the server and Docker is not. A refusal, such as GS-RUN-PODMAN_NO_USER, is shown on that page and Continue is not offered.

    What the installer configures

    For a Podman install, the installer does the following, and records what it created in /var/lib/gluesync/state.json so that the uninstaller removes only that.

    1. Packages (online only). The distribution’s podman and the tools of the Docker install (zip, jq). It then installs the optional crun, netavark, aardvark-dns, passt (and slirp4netns when passt is not available) one at a time: a package that does not exist on the distribution, such as passt on Ubuntu 22.04 and Debian 12, does not stop the install. On Ubuntu and Debian, uidmap is installed too. Docker Compose v2 is installed only when no Compose v2 is found: the docker-compose-plugin package from Docker’s repository, or the binary from GitHub on Amazon Linux, SUSE and openSUSE. Docker Engine (docker-ce) is never installed.

    2. Registries. /etc/containers/registries.conf.d/50-gluesync.conf with unqualified-search-registries = ["docker.io"], so that podman pull molo17/…​ works with a short name. On Red Hat family servers Podman refuses short names without it.

    3. Rootful. systemctl enable --now podman.socket podman-restart.service. The socket must appear at /run/podman/podman.sock within 30 seconds.

    4. Rootless. In this order, for the owner:

      • Sub-user and sub-group IDs. At least 65536 in /etc/subuid and /etc/subgid. When one is missing, the installer takes the first free range after the existing ones (at least 100000) with usermod --add-subuids …​ --add-subgids …​.

      • Lingering. loginctl enable-linger <user>, so that the containers of the owner keep running when nobody is signed in and start again at boot. It then waits up to 30 seconds for /run/user/<uid>.

      • Ports 80 and 443. /etc/sysctl.d/99-gluesync-podman.conf with net.ipv4.ip_unprivileged_port_start=80, loaded with sysctl --system. Nothing is written when the value is already 80 or lower.

      • Units. systemctl --user enable --now podman.socket podman-restart.service as the owner. The socket must appear at /run/user/<uid>/podman/podman.sock.

    5. Restart after a reboot on Podman 4. On Podman 4, podman-restart.service starts only containers with the restart policy always, and the kit uses unless-stopped, so the containers would stay stopped after a reboot. When the Podman major version is below 5, the installer writes a systemd drop-in, /etc/systemd/system/podman-restart.service.d/50-gluesync.conf (rootful) or /etc/systemd/user/podman-restart.service.d/50-gluesync.conf (rootless), that makes the unit start and stop unless-stopped containers too. Podman 5 needs no drop-in.

    6. The kit .env. These keys are written next to docker-compose.yml. Every other line is left as it is. On Docker the keys are not written, and an install that goes back to Docker removes them.

      Key Rootful Rootless

      GS_CONTAINER_RUNTIME

      podman

      podman

      GS_CONTAINER_SOCKET

      /run/podman/podman.sock

      /run/user/<uid>/podman/podman.sock

      GS_COREHUB_USERNS

      not written

      keep-id:uid=1017,gid=1017

      GS_CHRONOS_USERNS

      not written

      keep-id:uid=1017,gid=1017

      GS_CHRONOS_USER

      not written

      1017:1017

      GS_CONTAINER_SOCKET is the path of the socket on the host: the kit mounts it at /var/run/docker.sock inside the containers that need the engine (Conductor and the reverse proxy). GS_COREHUB_USERNS makes Core Hub run as user 1017 of the owner’s user namespace, so its data belongs to the owner and not to a number from the sub-ID range. GS_CHRONOS_USERNS and GS_CHRONOS_USER do the same for Chronos; Chronos also needs the user, because its base image names user 0, which keep-id alone does not override. gluesync start writes the keys again when they are missing or wrong.

    7. Ownership. The install folder belongs to the owner, and so do Core Hub’s and Chronos' data. On a rootless install, before every start, restart and update, Gluesync Control looks in data/core-hub, logs/core-hub, data/chronos and logs/chronos for files the owner does not own (written as container root by an older image, for example) and gives them back with podman unshare chown -R 0:0, logging one line.

    8. Check. podman info answers (as the owner, for rootless), the socket exists, Compose v2 is found, and, online, podman run --rm docker.io/library/hello-world runs. A failure of the last one is a warning.

    The installer does not touch /etc/docker/daemon.json, the docker group or the Docker service on Podman. It still creates the 10 GB swap file, and on Debian and Ubuntu /etc/sysctl.d/99-gluesync.conf, as on Docker. Log rotation is Podman’s own (containers.conf).

    With a docker command that is really podman-docker, an install that asked for Docker stops with GS-RUN-DOCKER_INSTALL_FAILED (exit 87), and the message says to use --runtime podman.

    Offline install with Podman

    The offline install works as on Docker (see Offline installation and updates) with two differences: no package is installed, and the engine must be there before you start.

    • Podman 4.9 or later, and Docker Compose v2, must already be installed. The socket may be stopped: the install enables it. When either one is missing, the install stops at once, before anything changes, with GS-RUN-102 (exit 83), and the message names what is missing.

    • Everything in What the installer configures except the packages is done: the registries file, the sub-IDs, lingering, the sysctl file, the units and the .env keys.

    • Every image of the platform archive is loaded with podman load (as the owner for rootless). Podman prints the names fully qualified, such as docker.io/molo17/x:tag. The installer reads them as molo17/x:tag, which is what the compose file names.

    sudo gluesync install --offline --runtime podman \
      --package gluesync-platform-linux-amd64-<version>.tar.gz \
      --kit-file <kit-id>-trial-kit.zip
    
    # Or with the offline script, which finds the two files next to itself
    sudo ./docker-installer-linux-offline.sh --runtime podman

    Manage Gluesync on Podman

    Use gluesync as on Docker. See Gluesync Control for Linux for every command.

    sudo gluesync start
    sudo gluesync status
    sudo gluesync restart
    sudo gluesync update
    sudo gluesync collect-logs
    • Who can run it. Root, or the owner of a rootless install, with or without sudo. Anyone else gets GS-CTL-007 (exit 228): this command needs root or access to Podman. A rootful Podman’s socket belongs to root.

    • Rootless. When root runs a command, Gluesync Control becomes the owner for every podman and Compose call. The owner can run it directly, with XDG_RUNTIME_DIR set (a normal login does that).

    • Status. gluesync status shows Podman : running (rootless) or Podman : running (rootful) where Docker shows Docker : running, and socket not running when the socket is down. gluesync status --json keeps the docker object and adds engine and mode. On a rootless install with a Core Hub older than 2.3.2.0 or a Chronos older than 0.3.3, status (also in --json, under warnings), gluesync start and the terminal screen show a warning. start is never blocked.

    • Socket down. start starts podman.socket for you and waits up to 120 seconds (systemctl start podman.socket, or systemctl --user start podman.socket as the owner for rootless). A user who is not root on a rootful Podman is told to run sudo systemctl start podman.socket.

    • Updates. gluesync update, pins and the downgrade backup work on Podman, online and with --package. Images are loaded with podman load and tagged with fully qualified names. See Platform updates. On a rootless install, an update that would leave Core Hub older than 2.3.2.0 or Chronos older than 0.3.3 is refused with GS-UPD-010 before anything changes.

    • Conductor. Gluesync Conductor detects Podman by itself, finds the host socket, and updates and restarts the stack through it. See Conductor on Podman.

    • Reboot. The containers start again after a reboot: rootful through podman-restart.service, rootless through lingering and the user’s podman-restart.service.

    Kits generated before Podman support

    A kit downloaded before Podman support mounts /var/run/docker.sock as it is, has no :z on its bind mounts, and has no userns_mode for Core Hub. On Podman with SELinux that makes Core Hub and Chronos exit with permission denied, and Conductor and the proxy get EACCES on the socket.

    Gluesync Control migrates such a compose file itself, on a Podman install only. It runs right after the kit is installed, and at every gluesync start, every gluesync restart and every gluesync update that applies (not --dry-run or --list):

    • every /var/run/docker.sock mount becomes ${GS_CONTAINER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock;

    • each service that mounts the socket gets security_opt: [label=disable] (the item is added to an existing security_opt, once);

    • every relative bind mount gets :z (:ro,z when it is read-only); named volumes, absolute paths and binds that already carry z or Z stay as they are;

    • gluesync-core-hub gets userns_mode: ${GS_COREHUB_USERNS:-};

    • gluesync-chronos gets userns_mode: ${GS_CHRONOS_USERNS:-} and user: ${GS_CHRONOS_USER:-}.

    Each rule is applied on its own, so a file migrated by an earlier Gluesync Control only gets what it is missing.

    Comments, order and indentation of the file are kept. The old file is kept once as docker-compose.yml.pre-podman, and one line tells what was changed. Docker installs are never touched. A kit made after Podman support has all of this already, and for it the migration changes nothing.

    SELinux

    On a server with SELinux enforcing (RHEL, Rocky Linux, AlmaLinux, Fedora), the kit works without turning SELinux off. It relies on two settings that are already in the kit’s compose file:

    • :z on every relative bind mount (:ro,z when read-only), so that Podman relabels the folder for the container. Docker accepts the flag and ignores it. It is never put on the socket.

    • security_opt: [label=disable] on the services that mount the engine socket (Conductor and the reverse proxy). The socket carries a label that confined containers may not use.

    If you add a service or a folder of your own to the compose file, give its bind mounts :z. Helper containers started by Conductor use label=disable themselves. getenforce prints the mode. The support bundle records it in system/selinux.txt.

    Manual setup without the installer

    The installer is the supported way. When policy asks you to prepare the server yourself, do the same steps by hand, then use Gluesync Control or Compose to start Gluesync.

    1. Install Podman 4.9 or later and Docker Compose v2 (docker-compose-plugin from Docker’s repository, or docker-compose-v2 on Ubuntu). Do not install podman-compose.

    2. Make short image names work with the Podman command line:

      echo 'unqualified-search-registries = ["docker.io"]' | sudo tee /etc/containers/registries.conf.d/50-gluesync.conf
    3. Enable the socket.

      Rootful:

      sudo systemctl enable --now podman.socket podman-restart.service

      Rootless, signed in as the owner:

      sudo loginctl enable-linger "$USER"
      systemctl --user enable --now podman.socket podman-restart.service
      grep "^$USER:" /etc/subuid /etc/subgid     # each must give at least 65536 IDs
      echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee /etc/sysctl.d/99-gluesync-podman.conf
      sudo sysctl --system
    4. On Podman 4, make the restart unit start unless-stopped containers (rootful: /etc/systemd/system/podman-restart.service.d/50-gluesync.conf; rootless: /etc/systemd/user/podman-restart.service.d/50-gluesync.conf), then run sudo systemctl daemon-reload (rootless: systemctl --user daemon-reload):

      [Service]
      ExecStart=
      ExecStart=/usr/bin/podman $LOGGING start --all --filter restart-policy=always --filter restart-policy=unless-stopped
      ExecStop=
      ExecStop=/bin/sh -c '/usr/bin/podman $LOGGING stop $(/usr/bin/podman container ls --filter restart-policy=always --filter restart-policy=unless-stopped -q)'
    5. Put the kit in the install folder (for example /opt/gluesync, owned by the owner for rootless) and add the three keys to the .env next to docker-compose.yml:

      # Rootful
      GS_CONTAINER_RUNTIME=podman
      GS_CONTAINER_SOCKET=/run/podman/podman.sock
      
      # Rootless (use the uid of the owner: id -u)
      GS_CONTAINER_RUNTIME=podman
      GS_CONTAINER_SOCKET=/run/user/1000/podman/podman.sock
      GS_COREHUB_USERNS=keep-id:uid=1017,gid=1017
      GS_CHRONOS_USERNS=keep-id:uid=1017,gid=1017
      GS_CHRONOS_USER=1017:1017
    6. Start Gluesync from the kit folder, with DOCKER_HOST pointing at the Podman socket. Run it as the owner for rootless:

      cd /opt/gluesync
      DOCKER_HOST=unix:///run/podman/podman.sock docker compose up -d                 # rootful
      DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock docker compose up -d     # rootless

      When there is no docker command (the Compose plugin is installed without Docker Engine, as the installer does), run the plugin binary directly, for example /usr/libexec/docker/cli-plugins/docker-compose up -d.

    Once the three keys are in the .env, gluesync start, gluesync status and the other commands recognise the Podman install even without state.json, and migrate an older compose file for you (see Kits generated before Podman support).

    Troubleshooting

    Installer codes

    The Podman install adds six codes, listed with every other code on Installer error codes and troubleshooting.

    Code Exit code What to do

    GS-RUN-PODMAN_NO_USER

    90

    Rootless needs an ordinary user. Run sudo gluesync install --runtime podman from that user’s own account, or add --rootful.

    GS-RUN-PODMAN_SUBID_MISSING

    91

    The owner has no range of 65536 IDs in /etc/subuid and /etc/subgid, and usermod could not add one. Add it by hand: sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 <user>, using a range that no other user has.

    GS-RUN-PODMAN_LINGER_FAILED

    92

    loginctl enable-linger failed, or /run/user/<uid> did not appear. Check that systemd-logind runs, then run the installer again.

    GS-RUN-PODMAN_SOCKET_INACTIVE

    93

    podman.socket could not be started, the socket file did not appear, or podman info does not answer. See The socket does not answer.

    GS-RUN-PODMAN_COMPOSE_MISSING

    94

    No Docker Compose v2 was found after the install. Install the docker-compose-plugin package from Docker’s repository, then run the installer again.

    GS-RUN-PODMAN_ROOTLESS_IMAGES

    100

    Rootless only: the kit has a Core Hub older than 2.3.2.0 or a Chronos older than 0.3.3, which cannot start as a non-root user. Nothing was started. Use --rootful or --runtime docker, or a kit with Core Hub 2.3.2.0 or later and Chronos 0.3.3 or later.

    An offline install without Podman or without Compose v2 is GS-RUN-102 (exit 83). A failed package command is GS-RUN-DOCKER_INSTALL_FAILED (exit 87), as on Docker.

    The socket does not answer

    # Rootful
    systemctl status podman.socket
    ls -l /run/podman/podman.sock
    sudo podman info
    
    # Rootless, as the owner
    systemctl --user status podman.socket
    ls -l /run/user/$(id -u)/podman/podman.sock
    loginctl show-user "$USER" | grep Linger      # Linger=yes
    podman info

    When the owner is not signed in and /run/user/<uid> does not exist, lingering is off: sudo loginctl enable-linger <user>. Work as the owner by signing in over SSH or at the console. With su or sudo -u, set XDG_RUNTIME_DIR=/run/user/$(id -u <user>) first, or systemctl --user and podman cannot find the user’s session.

    Containers are stopped after a reboot

    On Podman 4 this means the drop-in of podman-restart.service is missing (see What the installer configures). Check systemctl cat podman-restart.service (rootless: systemctl --user cat podman-restart.service), and for a rootless install that lingering is on. sudo gluesync start starts everything again.

    Permission denied on ports 80 and 443 (rootless)

    cat /proc/sys/net/ipv4/ip_unprivileged_port_start must print 80 or less. Create /etc/sysctl.d/99-gluesync-podman.conf as shown in What the installer configures and run sudo sysctl --system.

    Permission denied on folders or on the socket (SELinux)

    Look for AVC denials with sudo ausearch -m avc -ts recent. Check the compose file for :z on the bind mounts and label=disable on the services that mount the socket (see SELinux). A compose file from an older kit is migrated by gluesync start or gluesync restart.

    Files owned by a strange number (rootless)

    Files that a container writes as a user other than root show on the host with a number from the owner’s sub-ID range (for example 100999). Core Hub and Chronos are not among them on a current install (see the kit .env keys above), and Gluesync Control gives back their folders at every start. Read or change the others from inside the user namespace, as the owner:

    podman unshare ls -ln ./data
    podman unshare chown -R 0:0 ./data/<folder>

    Do not run chown as root on these folders from the host: it would give them to a user that the containers cannot map.

    A call is refused from a Podman network

    Podman’s default networks are 10.88.0.0/16 and 10.89.x.0/24, not the 172.x addresses of Docker. Use a version of the component that accepts private-network callers. Conductor and Chronos releases that support Podman do.

    Short image names fail with podman pull

    Pull with the full name (docker.io/molo17/…​), or make sure /etc/containers/registries.conf.d/50-gluesync.conf exists. Gluesync itself is not affected: Compose pulls through the API, which resolves short names to Docker Hub.

    The support bundle on Podman

    gluesync collect-logs collects the same entries as on Docker, with podman in place of docker, plus these (the files keep the docker/ and system/ folders of the bundle):

    • docker/podman-version.txt and docker/podman-info.txt;

    • docker/podman-ps-owner.txt and, for a rootless install, docker/podman-ps-root.txt: the containers of root, which shows containers that were started in the wrong place;

    • system/podman-socket.txt: systemctl status podman.socket podman-restart.service, as the owner for rootless;

    • system/podman-owner-session.txt: loginctl show-user <owner>, which shows lingering and the user manager;

    • system/podman-journal.txt (rootful): journalctl -u podman.socket -u podman.service;

    • system/containers-registries-<file>.txt, one for each file in /etc/containers/registries.conf.d;

    • system/selinux.txt: getenforce, always present;

    • a Container engine (Podman) section in system/system-info.txt: the runtime, the owner, the socket and whether it exists.

    The Docker service files (docker-service.txt, docker-journal.txt, docker-daemon.json.txt) are left out. Send the bundle to support as described in Enterprise support.

    Uninstall

    The Linux uninstaller, docker-uninstaller.sh, reads runtime in state.json and removes Gluesync from the engine it was installed on. For a rootless install it runs Podman and Compose as the owner.

    sudo ./docker-uninstaller.sh
    sudo ./docker-uninstaller.sh --purge-runtime    # also removes the Podman packages and, for rootless, the owner's Podman data

    It removes only what the installer created, as recorded in state.json: the registries file, the sysctl file (then sysctl --system), the podman-restart.service drop-in of Podman 4 (then a systemd reload), the sockets and podman-restart.service units that the installer enabled, the owner’s lingering, and, with --purge-runtime, the Compose plugin. A unit that was already enabled before the install is left enabled.

    • Sub-user and sub-group ID ranges stay: other software may use them. The script says so.

    • The uninstaller never deletes /var/lib/containers. To wipe the data of a rootful Podman, run sudo podman system reset yourself. With --purge-runtime, a rootless install removes ~/.local/share/containers and ~/.config/containers of the owner only.