Docker compose

To ensure a smooth experience in running Gluesync docker containers within docker-compose, please walk through the following checklist to ensure your environment is ready.

Following you’ll learn how a standard docker-compose.yaml file for Gluesync is composed, how to build/customize your own and how some of the Docker compose functionalities come to help to achieve your target goals.

A basic deployment for Gluesync is based out of at least three components:

With that minimum set of components you’ll be able to setup one pipeline and replicate as many entities (tables, collections…​) from source to target.

In case you want to run more pipelines you’ll need to add more source and target agents to your deployment.

You can browse our public Docker Hub repo to find the most recent image tag for each of our images.

Resource limits and provisioning are the perfect way to manage the resource sharing between each running container within your deployment.

By default each of our template comes with this values left commented, leaving the management of the resource sharing task to the docker engine itself; while this approach is good for POC and trials this is not recommended for production usage of the platform.

This two settings can be adjusted per each container by following this example:

Limits: use limits to limit the maximim number of resources (CPU cores & RAM) your container should use. Reservations: reserve and assign certain amount of resources (CPU cores & RAM) to your container to use when needed, else will be shared with others.

By default the container runs under the UTC time zone, used for standard time reference. You’ll then see console logs being served in UTC by default and that might be required to be changed to ease the tasks of troubleshooting and log ingestion.

To change the default time zone reference you can use this snippet, to be placed within each service:

For your convenience you can pick up the proper time zone by looking at the list available at the following link: time zones list.

Time zone can so be changed by specifying a different value insted of the Etc/UTC present in the given example, like America/Detroit.

By default our template kits (like the Gluesync Trial Kit) come with peristency, even though they are meant for a trial and disposal of the environment they rely on the physical storage provided by the Docker engine where otherwiser a docker compose down command would tear down and clean up the container status, removing ephimeral volumes all together to the instances.

To enable/disable persistency you can comment out the few lines from our template docker-compose.yaml file or add the following like in the given example below:

As you can see from the above example, a phyisical volume is mapped to the folder where the actual application data is located. By removing this line entry the volumes belonging to this container will be mamaged by the Docker engine as ephimeral.

Networking in Docker requires some knowledge of computer networks but we try here following to give you some basic commands that are often coming to be handy when working with Docker containers and having to deal with the reachability of host’s hosted DBs or intra-hosts container reachability.

Make sure you have Docker up and running, then issue the following terminal commands:

Ability to get the set of logs redacted by the docker engine itself comes really to be handy when talking with the support.

Docker logs can be easily exported by issuing the following command:

This will export the full set of logs belonging to the Docker compose deployment in one single file named logs.txt than can be then zipped and uploaded to our support channel.

Should you need to use a custom registry, e.g. because your environment cannot reach docker hub’s container registry, please contact us to arrange a specific setup. You can use the provided Access Tokens to locally mirror the container.