A GitOps Deployment with Flux on DigitalOcean Kubernetes – InApps 2022

GitOps is gaining momentum as the preferred mechanism for continuous deployment. Based on the demo that I showed during the DigitalOcean Deploy Conference in November 2021, I am bringing you the step-by-step GitOps tutorial to perform deployments at scale.

We will manage the deployment of a simple web application across three different DigitalOcean Kubernetes clusters running in Bangalore, Singapore, and London regions. The Weaveworks-managed open source Flux, the most popular CD tool will be used to bootstrap GitOps, configure Ingress infrastructure, and finally deploy the application.

We will follow some of the best practices of GitOps for this tutorial such as leveraging git as the single source of truth, using git workflow to deploy infrastructure and applications.

Except for the bootstrapping, we will always use the git command and not rely on the CLIs (fluxctl and kubectl) for the configuration. We will incrementally add an ingress component (infrastructure) packaged as a Helm Chart and a web application (apps) declared in a set of YAML files to the repo, which will be eventually reconciled across all the clusters.

Whether it is one cluster or tens of thousands of clusters, the workflow remains the same. The objective of this tutorial is to show at-scale deployment targeting multiple Kubernetes clusters.

Prerequisites

Step 1: Preparing the Environment

Let’s begin by launching the Kubernetes clusters with doctl CLI. By the end of this step, we should have three clusters running in three different regions of the Digital Ocean cloud platform. Once the clusters are provisioned, we will download kubeconfig and rename the context.

Next, create a GitHub repository that will be used for storing the artifacts. Let’s call this do-gitops-demo.

Assuming you have created a GitHub personal access token, set the environment variable for Flux to access the repository.

export GITHUB_TOKEN=YOUR_GITHUB_PERSONAL_ACCESS_TOKEN

Step 2: Bootstrapping Clusters for GitOps

Let’s make sure that the cluster is ready for Flux. We will do this by running the command, flux check.

Let’s bootstrap the clusters by pointing them to the GitHub repository created in the previous step.

This results in two things — Flux adding the manifests to the fleet directory of the repo and reconciling the cluster with the manifests.

The directory structure looks like this:

You can verify the flux-system namespace on each cluster. This confirms successful bootstrap.

We are now ready to configure the infrastructure components (Ingress) and deploy them through the same workflow.

Step 3: Deploying Ingress through GitOps

With the Flux GitOps agent running in all the clusters, we can leverage it to push newer deployments by committing the artifacts to the Git repository.

In this step, let’s create a namespace ingress-system and install the NGINX ingress Helm Chart. Let’s leverage the helmrepositories and helmreleases CRDs added by Flux.

Start by cloning the Git repository to your local workstation.

git clone https://github.com/$OWNER/do-gitops-demo.git.

At this point, this repo contains the YAML artifiacts responsible for deploying Flux operator. We will add a new directory, infrastructure and commit the YAML files with Helm Chart for NGNIX-based ingress.

cd do-gitops-demo
mkdir infrastructure

Let’s add the YAML files to the directory.

These three files roughly translate to the same sequence of commands run to install NGINX ingress:

kubectl create ns ingress-system

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

helm install --namespace ingress-system
nginx-ingress ingress-nginx/ingress-nginx

Finally, we need to add a Kustomization to the cluster directory under fleet which will act as a pointer to the infrastructure components. Since the directory, fleet is registered with the Flux agent during the bootstrapping, that’s the best location to add the below YAML file.

For example, to target the Bangalore cluster, copy the below file to fleet/blr1 directory. This approach gives us a chance to customize the artifact per cluster at a later stage.

Add the file to all the clusters to ensure that they deploy the ingress.

Notice the addition of infrastructure.yaml in each cluster-specific directory and the infrastructure directory with the YAML artifacts.

It’s time to commit the new artifacts and push them to the remote GitHub repository.

git add .
git commit -m "Added infrastructure components"
git push

The moment the changes are committed, Flux agent starts the reconciliation process. In a few minutes, you should see all the clusters create the ingress-system namespace.

Since the creation of an ingress results in provisioning a load balancer, the Digital Ocean cloud console shows that there are three load balancers created per cluster.

This step demonstrated how to deploy a Helm chart through the GitOps workflow. Let’s move to the next step where we will deploy the web application that leverages the ingress created in this step.

Step 4: Deploying a Web App through GitOps

For the web application, let’s follow the same steps of adding the manifests to the apps directory and then adding a Kustomization file to the fleet directory.

cd do-gitops-demo
mkdir apps

Inside the apps directory, create the below YAML files:

Finally, add the below Kustomization file to all the clusters under the fleet directory.

With the addition of apps, notice how the directory structure has changed.

git add .
git commit -m "Added web application"
git push

In a few minutes, you will see the mywebapp namespace with the web application.

Access it by visiting the load balancer IP address to see the below web page:

Edit the do-gitops-demo/apps/web.yaml to update the image tag from v1 to v2.

Commit the code and push it to the remote GitHub repository. Access the load balancer IP again to see the V2 version of the web application.

If you revert the last commit, you can roll back the deployment to V1. This is the beauty of using GitOps.

This completes the step-by-step guide to using FluxCD with DigitalOcean Kubernetes. In the upcoming tutorials, we will explore how to use Kustomization to customize the deployment per each cluster. Stay tuned.