Policy-Based Compliance the GitOps Way – InApps Technology 2022

A key priority for SecOps teams is to verify that policies and other guardrails are present for every release. In a world of increased automation, this is becoming a challenge as operations are automated at the expense of compliance. In this post, we look at a modern, declarative and policy-based approach to building compliance into each step of the system. We describe an effective method to manage change control over software delivery practices and put more control into the hands of compliance and SecOps teams.

An execution engine is defined by a YAML config file. This is arguably one of the most overextended and misused parts of a software delivery toolchain. It is most likely the biggest source of headaches for anyone looking after compliance, since the abuse these config files suffer will likely leave no way to either troubleshoot or audit the process.

But what if a definition of the policy or policies governing this config file could be set in stone? What if this policy would be the filter that would leave out any noncompliant deployment? What if that policy could be modified by anyone, as long as the compliance team approves of it? These scenarios are possible in the declarative world of Kubernetes.

Proof Points for Compliance

First, let’s look at how this entire process works. The key idea behind such a policy-based system is to have a clear “proof point” for each policy. This includes checks at the following critical stages of the software delivery pipeline:

1. In code
2. In config
3. In the pipeline preflight checks
4. In the post-deploy runtime checks

If implemented in this way, it would show that safe delivery is assured end to end. It would guarantee the integrity of the code being released.

Now, to get into the nuts and bolts of how this actually works in a modern system comprising git repositories, continuous integration/continuous delivery (CI/CD) pipelines and Kubernetes-managed infrastructure.

The Role of Git in Compliance

Policy can be defined in a declarative format and stored in code in a specific git repo. This is a crucial step, as git brings with it powerful capabilities for change management such as version control and fine-grained access controls. Such a repo would provide only admin privileges to the compliance team. However, as with any git repo, source code stored in it is open to pull requests from anyone within the organization. This gives development teams the autonomy they need, while not sacrificing on the level of control that compliance teams need. These policies can be implemented in any execution engine, be it a CI one or a CD one.

Leveraging git for policy enforcement addresses the first two steps of having proof points in code and in config. Now that we have the git part of the process in place, let’s look at the role of the execution engine.

Weave GitOps as the Execution Engine

Suppose that a core template set is stored in a compliance repo to reduce manifest maintenance. The next step is to have the execution engine apply this template set pre-deployment. There are many options for an execution engine, but for the purposes of this post, we’ll discuss Weave GitOps. At runtime, Weave GitOps can be configured to fetch and deploy a specific subset of the manifest from within the compliance repo. Weave GitOps will then deploy the policy contained in those manifests, which in turn would define environment config and access permissions, among other policy variables already defined. This can be executed on a single Kubernetes cluster or multiple target clusters running anywhere from on-premises, to cloud, to edge.

Managing Drift in Production

With this pre-deployment policy display, we are assured that nothing can run against it. But even in a post-deployment stage, changes can be introduced that go against the set compliance policies. End-to-end compliance needs to take into account the production environment as well.

Here again, Weave GitOps plays a key role in preventing drift. Weave GitOps continuously performs health checks that map what’s running in the target environment to the predefined policy in the compliance repo. Weave GitOps then sets off an alert if any drift is detected, pinpointing the exact change in the Kubernetes cluster. If the change is legitimate, it would require a simple edit to the policy template in git to accommodate this change in the future.

In many cases, the drift needs to be corrected. Here, Weave GitOps is able to either roll back to a previously compliant state or override any drift that manual changes would have enforced. This is where git’s version control is indispensable. Weave GitOps allows you to pick a previous compliant state — whether that’s one pull request or tens of pull requests ago — and roll back to that specific state.

Conclusion

As automation has greatly advanced the pace and frequency of deployments, compliance teams have struggled to keep pace. This compromise can no longer be ignored. As the software stack has become increasingly declarative, thanks to Kubernetes and git, compliance should also become declarative.

Enforcing this sort of compliance requires git for its version control, access controls and collaborative capabilities. Also, it needs to be augmented with an execution engine, like Weave GitOps, for control over the pre- and post-production phases. When implemented in this manner, trusted delivery becomes the norm.

Photo by Mikhail Nilov from Pexels.