GitOps is a modern paradigm to simplify and automate CI/CD pipelines for software applications. As an acronym, GitOps is the combination of two different words: Git + Operations.

In essence, your infrastructure code in Git directly represents the actual state of your infrastructure environment. Any changes made in Git are immediately reflected in your infrastructure resources. There is no intermediary code execution layer to interpret your changes; Git essentially acts as an extension of your cloud infrastructure's configuration.

This means, that if it's not in Git, it does not exist.

The essential principles

The main ideas and benefits around GitOps are:

  • Asynchronous infrastructure state synchronisation: Ensures that your infrastructure state is continuously aligned with the desired configuration.
  • Forced infrastructure as code: All infrastructure configurations, including release versions, should be defined as code, a practice often overlooked in traditional CI/CD.
  • Modularisation: Encourages breaking down infrastructure into reusable modules for easier management.
  • Unconditional automation: Reduces manual intervention, minimising errors and speeding up deployments.
  • Developer empowerment: Developers’ applications source code includes manifests and configuration files that declaratively specify the desired state of the infrastructure. This encourages the adoption of DevOps.
  • Separation of concerns: Continuous Integration should be separated from Continuous Delivery pipelines, this brings great security benefits.

A simple Git workflow, git add → git commit → git push, will suffice to trigger a GitOps synchronisation and get your repositories in-sync with your Kubernetes cluster and workloads.

Advanced controls like change requests or auditing are already built into your version control software, such as through pull requests or the Git log.

Throw out your CI/CD pipelines

The typical CI/CD approach is easy to implement. Let’s take GitHub Actions as an example; developers place a script at the root of a project which triggers a build via a hook whenever an event you specify in that script occurs. By enabling a Cloud Build plugin in the same repository, a simple git push command from an authorised terminal will perform CI/CD in one single step.

However, you pay a cost for that simplicity and that is in terms of diminished security and unexpected downtime in case of disaster.

Lack of security

With this approach, your CI tooling pushes and deploys images to the cluster. For the CI system to apply the changes to a cluster, you have to share your API credentials with the CI tooling. That means your CI tool becomes a high-value target. If someone breaks into your CI tool, they will have total control over your production cluster, even if your production cluster is highly secure.

Disaster recovery nightmare

What happens when you need to recreate your cluster in the case of a total meltdown? How do you restore the previous state of your application? You would have to run all of your CI jobs to rebuild everything, and then re-apply all of the workloads to the new cluster. The typical CI pipeline doesn't have its state easily recorded like when you're using GitOps.

Why its worth the cost

The GitOps agent is the pivotal junction that allows synchronising state between Git and your Kubernetes Cluster. These agents are in essence Kubernetes Controllers that implement a watch loop over the resources they control, continuously observing and acting upon the actual versus desired system state.

Separation of concerns

GitOps separates CI from CD and that will improve your security posture at the expense of additional complexity during the initial setup. Remember that GitOps is a cloud-native architectural pattern which is fully compatible with the declarative nature of Kubernetes. We consider DevOps without GitOps to be an anti-pattern in Kubernetes deployments.

CI tooling:

  • Builds run outside the production cluster, only the CD tooling runs inside it.
  • Because builds are triggered from source code, the CI tooling must have READ access to the source code plus READ/WRITE access to the container registry where the build artefact is pushed.

GitOps agent:

  • Runs inside the production cluster so it has READ/WRITE access to it.
  • Must READ the image registry.
  • READ/WRITE access to configuration repository.

Important considerations for adoption

The principles seem sound and the benefits of using GitOps are clear. However, getting things right isn’t black and white and implementations aren’t immediately obvious.

GitOps is an architectural pattern that requires a shift from the typical CI/CD pipeline “one-click, one cloud plugin” paradigm.

The market for GitOps agents is in full swing and not all agents are created equal. It is important you weigh pros and cons based on your specific needs.

What are the main stumbling blocks?

Time: Implementing GitOps is likely going to require you to rewrite a large portion of your current infrastructure codebase.

Lack of internal capabilities: Limited in-house expertise can hinder the adoption and smooth operation of GitOps within smaller teams.

Risk of configuration drift: GitOps configurations often require careful tweaking to ensure smooth synchronisation between your Git repository and your Kubernetes cluster.

Security awareness and implementation: The new way of thinking of CI/CD requires a new way of thinking about securing deployments and configuration changes.

Lack of observability: Ensuring that your GitOps agent performs as expected requires oversight to your GitOps synchronisation health.