Kubernetes

Kubernetes (often abbreviated as K8s) is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications. Since all the Gluesync components are shipped as docker images, it is possible to leverage them as best and run the entire system in a Kubernetes cluster.

The first requirement to run a Kubernetes application is to have a Kubernetes cluster. In this tutorial both minikube cluster and GKE will be taken into account. Other types of Kubernetes clusters are outside the scope of this document, while fully supported by Gluesync.

Setup of minikube

To setup minikube (a simple Kubernetes cluster for development purposes) on your machine, you can follow the instructions at this page, making sure to select the correct architecture of your machine.

Once installed, to start the minikube cluster you can run:

minikube start

Install Kubectl on your machine

Kubectl is the command-line tool used to interact with a Kubernetes cluster. It is an essential utility for managing and troubleshooting Kubernetes resources, enabling users to perform a wide range of operations on the cluster, including deploying applications, inspecting resources, scaling workloads, and much more.

Follow this page to install kubectl on your machine.

Setup for gcloud for GKE development

In case you have decided to deploy your application on GKE, you can follow along to install and configure the components that are necessary for this setup.

Install gcloud SDK

To install gcloud on the host machine, follow these steps on this page or use the brew command line utility if you are working on a mac machine.

brew cask install google-cloud-sdk

Once the sdk has been installed, run the following command to initialize it, you will be prompted to provide your login details and project information:

gcloud init

In order to ease the management of the GKE cluster, another couple of gcloud components must be installed: the gke gcloud auth plugin and the kubectl component.

The gke gcloud auth plugin can be installed with the following command:

gcloud components install gke-gcloud-auth-plugin

and the kubectl component with:

gcloud components install kubectl

Kubectl is a command-line tool that you can use to interact with your GKE clusters, while the gke-gcloud-auth-plugin extends kubectl’s authentication to support GKE.

To configure kubectl to work with your cluster you can follow the steps shown by pressing the connect to cluster button on your google cloud GKE page for your cluster. Ideally, you will be given a command similar to the following:

gcloud container clusters get-credentials <your_cluster_name> --region <your_cluster_region> --project <your_project_name>

Kubectl can be used to manage both clusters, minikube or GKE at the same tiem, it is only important to make sure that when launching the commands, the correct cluster context is selected. This can be checked with the following command:

kubectl config get-contexts

Getting Started with Helm

One of the easiest ways to manage and deploy applications in Kubernetes is by using the helm package manager which simplifies the development and deployment steps of the application.

In this tutorial it will be shown how to organize the application in the form of a helm chart and to install it on the Kubernetes cluster, whether it is minikube or hosted on the GKE platform.

Helm install

To install helm follows the steps that are suited for your operating system, more information is available at this website.

Write your first template

Once helm has been installed, to create a helm chart you can run the following command:

helm create <your_chart_name>

this, will create a folder with the following structure:

mychart
├── Chart.yaml
├── charts
├── templates
│   ├── NOTES.txt
│   ├── _helpers.tpl
│   ├── deployment.yaml
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── service.yaml
│   ├── serviceaccount.yaml
│   └── tests
│       └── test-connection.yaml
└── values.yaml

The most important piece of the puzzle is the templates/ directory. This is where Helm finds the YAML definitions for your Services, Deployments and other Kubernetes objects. If you already have definitions for your application, all you need to do is replace the generated YAML files for your own. What you end up with is a working chart that can be deployed using the helm install command.

It’s worth noting however, that the directory is named templates, and Helm runs each file in this directory through a Go template rendering engine. Helm extends the template language, adding a number of utility functions for writing charts. The values.yaml file contains the values that are passed into the chart. For more examples and details about helm charts follow the documentation on the website.

Gluesync in its smallest installation consists of at least three main components. The first is the core hub which is the main component of the application and is responsible for the orchestration of the agents, while the other two are agent components (one for the source and the other for the target database). A simple example of a Kubernetes service of the core hub component may look as follows:

apiVersion: v1
kind: Service
metadata:
  name: {{ include "gluesync.fullname" . }}-core-hub
  labels:
    app.kubernetes.io/name: {{ include "gluesync.name" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}
spec:
  type: {{ .Values.service.type }}
  ports:
    - name: port-1717
      protocol: TCP
      port: 1717
      targetPort: 1717
  selector:
    app.kubernetes.io/name: {{ include "gluesync.name" . }}-core-hub
    app.kubernetes.io/instance: {{ .Release.Name }}

In Kubernetes, Services are a key abstraction that provides stable networking and load-balancing for Pods. Services enable communication between different components of an application or with external clients, abstracting away the dynamic nature of Pods. Pods are the smallest and most basic deployable units. A Pod represents one or more containers that are tightly coupled and share the same execution environment. Pods are designed to host one or more containers that work together as a single cohesive unit.

The life cycle of a Pod is usually controlled by a Deployment, which is a higher-level abstraction used to manage and control the lifecycle of Pods and ReplicaSets. In Helm charts, Deployment, Pods and Services as well as other kubernetes components, are usually linked through the use of selectors.

Selectors are used to define how Kubernetes resources match or associate with Pods based on their labels. Selectors are essential for Helm charts because they ensure that the Helm-defined Kubernetes objects interact correctly with each other and are properly scoped within the application.

An example of a deployment template may look as follows:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "gluesync.fullname" . }}-core-hub
  labels:
    app.kubernetes.io/name: {{ include "gluesync.name" . }}-core-hub
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/version: {{ .Chart.AppVersion }}
    app.kubernetes.io/managed-by: {{ .Release.Service }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app.kubernetes.io/name: {{ include "gluesync.name" . }}-core-hub
      app.kubernetes.io/instance: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app.kubernetes.io/name: {{ include "gluesync.name" . }}-core-hub
        app.kubernetes.io/instance: {{ .Release.Name }}
    spec:
      containers:
        - name: gluesync-core-hub
          image: "{{ .Values.gluesyncCoreHub.image }}"
          ports:
            - containerPort: 1717
              protocol: TCP
          volumeMounts:
            - name: gluesync-license
              mountPath: /opt/gluesync/data/gs-license.dat
              subPath: gs-license.dat
            - name: gluesync-core-hub
              mountPath: /opt/gluesync/database
            - name: gluesync-bootstrap
              mountPath: /opt/gluesync/data/bootstrap-core-hub.json
              subPath: bootstrap-core-hub.json
      volumes:
        - name: gluesync-license
          configMap:
            name: gluesync-license
        - name: gluesync-bootstrap
          configMap:
            name: gluesync-bootstrap
        - name: gluesync-core-hub
          persistentVolumeClaim:
            claimName: gluesync-core-hub

In this deployment you can easily see how Deployment labels match with the Pod template.

The Pod spec can contain one or multiple container definitions. Each container definition will have their env variables, ports and volume mounts definitions as well as the docker image, which for this case, it will be fetched from the molo17 docker hub repository.

Volume mounts specify how and where a container accesses the volume defined at the Pod level. Again the match between volumes and volume mounts is done through their name (visible in the example). For further customization and other Kubernetes concepts rely on its documentation.

Install helm chart on the clusters

Once the helm chart is ready and all the templates are completed, it can be installed on the cluster through the helm command line utility by launching a command from within the helm directory.

Here is an example:

helm install <chart-name> ./ --values ./values.yaml

This command will install the helm configuration into the cluster.

Helm uses the same Kube configuration file and looks for it in the same location as kubectl, therefore the same configuration applies.

For a complete example of a helm chart of the main Gluesync components, have a look at this repository.