Argo, the Kubernetes-Native Workflow Engine, Joins the CNCF – InApps 2022

Earlier this month, the Argo Project, a container-native workflow engine for Kubernetes to deploy and run jobs and applications, joined the Cloud Native Computing Foundation (CNCF) as an incubation-level hosted project.

By joining the CNCF,  the Argo project hopes to more closely work with a number of projects that are already members of the foundation, and to “empower organizations to declaratively build and run cloud native applications and workflows on Kubernetes using GitOps,” Intuit VP of Product Development Pratik Wadher said in a statement.

Argo was originally created in 2017 at Applatix before being acquired by Intuit, and the project already integrates with or uses a number of CNCF projects, including gRPC, Prometheus, NATS, Helm, and CloudEvents. The four sub-projects that comprise Argo include Argo Workflows, a  container-native workflow engine for Kubernetes supporting both DAG and step-based workflows, Argo Events, an event-based dependency manager to trigger workflows and applications, Argo CD, which supports declarative GitOps-based continuous delivery of any Kubernetes resource, and Argo Rollouts, which provides declarative progressive delivery strategies such as canary and blue-green deployments.

Wadher explained in an email how teams at Intuit pieced together tools before Argo to try to create the same effect.

“We had disparate deployment tools across the many business units and there was no standard method of deploying software across Intuit. Some development teams used CI tools like Jenkins to ‘script’ their deployment process and others used Spinnaker while yet others used homegrown tools and stitched multiple pieces together,” wrote Wadher. “Obviously, these disparate tools didn’t work and, at best, provided a sub-optimal developer experience as we moved to a platform based on Kubernetes. For example, these approaches would have required giving Jenkins access to production systems creating security risks and making the deployments complex and error-prone.”

While there might be a desire to compare Argo to a continuous integration, continuous delivery (CI/CD) pipeline, Wadher explained that it is “much more” than that, and that it is “unique in providing a set of Kubernative-native GitOps-based tools that integrate deployment and running of jobs, events and services into a single framework.” With its included single sign-on (SSO), role-based access control (RBAC), and audit and security functionality, Argo tailors its capabilities for enterprises running large, multicluster Kubernetes deployments, explained Wadher. Furthermore, its scalability, DAG support, and artifact management, he sais, have made it “very popular in the ML and Data community.”

Specifically, Argo was created to be a Kubernetes-native tool that would use Git as the single source of truth, wherein developers could push commits and avoid manual deployments, as well as the errors that come alongside them.

“Developers can deploy their application to multiple clusters and monitor and troubleshoot their applications centrally without logging into Kubernetes. Furthermore, Argo CD automates the rollbacks of applications to a previous state as well,” writes Wadher. “That greatly reduces the burden on the developers and they are now much more confident in pushing changes to production in the middle of a day. With Argo CD, our developers, on average, do more than 200 deployments a day to production compared to each deployment taking multiple days before. ”

Looking ahead, Wadher said that the future for Argo would largely be decided by the community, but that MLOps and continuous and progressive delivery would be the two main areas of focus. For MLOps, he expects a better integration between Argo Events and Workflows, with better monitoring and reporting. On the other point, he said the project plans full support for progressive delivery, with integrations for numerous service meshes, as well as the extension of the “GitOps paradigm” to manage all computational methodologies including, batch, event-based and services, as well as control plane elements in Kubernetes.