Flux Takes Its Continuous Delivery and Operations to CNCF Incubation – InApps Technology 2022

The Flux project has moved on to the incubation tier of the Cloud Native Computing Foundation (CNCF), bringing its unique form of continuous delivery to the foundation’s second tier of project maturity. Having joined the CNCF in August 2019, Flux has since added seven maintainers and increased adoption across more than 80 organizations in production, all the while working on a complete refactoring of the application to more loosely couple the various tools that comprise Flux.

While Flux bills itself as a continuous delivery tool, it also flips the script for traditional definitions of continuous delivery, with Flux creator and maintainer Michael Bridgen crediting Flux as being “the project that gave rise to ‘GitOps’.” GitOps, simply put, is a paradigm wherein operations and infrastructure are included in the continuous integration and delivery (CI/CD) process.

Traditionally, CI/CD exists as a linear process, where an event occurs, kicking off a pipeline in response, that terminates with the delivery of software to an infrastructure. For many, that event is a developer committing code, which then initiates a build process, ending with a container getting deployed to Kubernetes. What happens, though, when the infrastructure that software is deployed to gets changed? Flux addresses this issue by not only responding to events during the integration process but also at the other end of that normally linear process, reconciling builds when changes occur to infrastructure as well.

Weaveworks Chief Technology Officer Cornelia Davis credits Kubernetes as really shifting this paradigm, with its declarative model, and Flux similarly practices a reconciliation-based form of continuous delivery and operations.

“Kubernetes really popularized this notion of you’re never done. Things are done not as a result of an event, like somebody checking something into a Git repository, but things are changing all the time and you constantly have to be responding to these changes. It could be that somebody checked something into a repository, it could be that something in the infrastructure has changed, and now that delivery that you did to the running environment needs to be adjusted,” explained Davis. “One of the things that was a bit of a game-changer for Flux is this notion of a reconciliation-based way of doing continuous delivery. Flux really transformed this idea around continuous delivery as something that happens as a reconciler.”

The original Flux consisted of a number of tools in one monorepo, plus extensions to cover GitOps for Helm. Recently, these tools have been refactored  into the GitOps Toolkit, a set of APIs and controllers that capture the capabilities of Flux in a set of composable modules. Flux v2 then provides a composition of those modules into a package that has feature parity with Flux v1, including the Helm capabilities. Additionally, the Flux project now also includes, Flagger, a progressive delivery tool for Kubernetes. Flux v2, Davis says, has feature parity with v1, minus “a few tiny things that we deprecated because if you did things in that way, you’d end up hurting yourself.”

Operationally, the primary change with Flux v2 will be how Flux is installed into your Kubernetes clusters, said Davis, with Flux now consisting of a suite of controllers instead of a single controller. This loose coupling of services will allow for a greater level of configuration, especially for those running Flux at scale. Only minimal changes to Kubernetes workload configurations being managed by Flux are necessary in the move from v1 to v2, and the Flux community has produced detailed guides to assist.

“GitOps is being used as a really important enabler of ‘at scale’ and by ‘at scale’ I don’t mean necessarily hundreds of clusters, I mean tens of thousands, or hundreds of thousands of clusters,” said Davis. “We’re seeing telco companies that are putting Kubernetes on cell towers, and we work with a fair number of them at Weaveworks, and those systems might be touching Git repositories more repeatedly. Performance is likely going to benefit from the loose coupling.”

Flux also offers an SDK  for creating additional Flux controllers, and Davis said that the project hopes to include many other projects, beyond Flagger, moving forward.

“We are elevating this notion now of GitOps pipelines, where you can now assemble those reconciliation-based ways of doing things into these GitOps pipelines that take things from continuous delivery all the way out to operations,” said Davis. “What’s going to ultimately end up happening is that we are going to add additional controllers, GitOps controllers that are going to generate an ecosystem of controllers. Flux itself is building up the suite of controllers that serve the continuous delivery and continuous operations needs, so you’ll start to see a suite of controllers, as well as tooling that allows us to stitch those controllers together into these continuous delivery and continuous operations pipelines that we call GitOps pipelines.”