Cloudify and Spot.io — Cloudify to manage full life cycle of Kubernetes Ocean instances for cost optimization

Introduction

Today’s agile development requires rapid creation of Kubernetes clusters. This covers everything from development to testing and production.

Proper creation and deployment of a Kubernetes cluster is complex, requires multiple resources and costly. Moreover, customers have multiple Kubernetes flavors and they want to manage them in the same way.

Cloudify manages different Kubernetes clusters in the same way, consistently, whether it’s a managed EKS such as EKS, AKS, or GKE cluster and unmanaged clusters such as kOps, Kubeadm or Kubrspray.

Spot Ocean integrates seamlessly with and supports all k8s distributions to manage their worker nodes. This includes managed clusters such as EKS, AKS or GKE and unmanaged clusters such as kOps, Kubeadm or Kubespray. Spot.io Ocean runs on spot instances, automates both infrastructure and pod right-sizing and cuts costs substantially, while providing enhanced automation and visibility into the k8s clusters.

In addition, all this process could be triggered from GitHub utilizing GitActions workflows.

Full Consistent Lifecycle Management

As we can see in the image above, you choose your Kubernetes distro from a marketplace and decide whether you want to connect it to Spot.io Ocean for cost optimization.

The next step would be to review the Terraform plan before resources are created.

After approving the Terraform plan, you apply it. This is the actual deployment of resources. In this example we used already existing Terraform modules from the Terraform Registry.

Cloudify, as the orchestrator of orchestrators, knows to take Terraform resources and deploy them as a Cloudify blueprint. The same is true for Cloud templates like CFT and ARM.

Now the cluster is created and you can see it in the figure below

After the cluster is created, it is connected to the Spot.io console, utilizing the right token as can see in the Ocean dashboard below

You can view the Ocean cluster history over time and additional KPIs.

Let’s view it now in Cloudify

You can see the EKS-by-SPot-Ocean blueprint in the figure above. And if we look at the actual deployment we can see the topology and the execution graph.

Kubernetes Environments as a Service

Cloudify manages different Kubernetes flavors consistently. You can add your own resources to an environment that you create. You can also tag it with a label. Later on Cloudify can perform an operation on all Kubernetes clusters that match a specific label. A label could be for example DEV, PROD or US-EAST, US-WEST.

Managing the life cycle of an environment, in our case Spot.io Ocean, means configuring, provisioning, starting, stopping and deleting the environment.

Cloudify exposes the environments end points, meaning the endpoints you can interact with the environment. In Kubernetes it exposes the Kubernetes API endpoints, so one can deploy services and application pods on it. It could be regular application pods or a complex stack like frontend, application and a database,e.g. WordPress or Drupal.

GitActions

Kubernetes environments, and in this case Kubernetes Ocean clusters, can be provisioned utilizing GitActions. For example, upon a Git merge you can initiate a system test to deploy a Kubernetes Ocean environment, provision the application pods on top and run system tests.

Video: A full video of the whole process could be found here.