Add cluster to STOIX

In this quickstart, you schedule and run a simple workflow job. You will learn how to easily add your Kubernetes cluster to STOIX and how to run your job on a recurring time schedule.

Kubernetes clusters play an important role in running jobs managed by STOIX. With the help of STOIX and Kubernetes you can as easily run a few recurring jobs as well as scale up to thousands of jobs without needing in-depth knowledge of infrastructure nor DevOps. You can easily add one or more clusters from your cloud providers or on-premises and choose where your jobs execute.

Pre-requisites

You will need to connect to a Kubernetes cluster. Follow one of the following guides for your cloud provider to set up a basic cluster and on how to get access. You can easily test if you have authenticated to the cluster by trying to check machine information through the terminal command kubectl get nodes.

Google Cloud

• https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-zonal-cluster

Amazon Web Services

• https://docs.aws.amazon.com/eks/latest/userguide/create-cluster.html#aws-management-console
• https://docs.aws.amazon.com/eks/latest/userguide/create-kubeconfig.html

Microsoft Azure

• https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal

STOIX utilizes a lightweight controller running in your cluster to easily manage your jobs. It makes sure jobs run when intended and reports metadata back so you can access information in the dashboard in a secure way. The data management and controller’s access is further described under the section “Concepts”. With a new cluster set up, the next step is to add the cluster to STOIX.

Add a cluster to STOIX

Visit the STOIX dashboard under Clusters in the menu. Click the plus sign at the top right to add a new cluster. Provide a name for the cluster, such as “Production data engineering jobs”. Optionally add labels to provide extra information, such as “country = sweden”. Labels will in the future help you further get insights in your data set-up. Then click “Create” and you are now presented with the cluster information display. Download the “Kubernetes configuration” and deploy the controller by applying the configuration with the following terminal command: kubectl apply -f path/to/stoix.yaml. The content of the yaml is further described in the Concepts section.

As soon as the controller is set up correctly, the yellow bar on cluster information in the dashboard should become green and say “Cluster is connected”.