CircleCI Containerizes Build, Test and Deploy Services as Separate Components – InApps 2022

Continuous integration and delivery platform provider CircleCI has unveiled one of its most-asked-for features — the decoupling of build, test, and deploy stages of software development.

“Previously we supported jobs being a build in its entirety. Now you can break all that up into component jobs, which is great, especially when you have customers moving from more monolithic approaches to microservices and container-based environments where you have lots of services being meshed together,” said Jim Rose, CEO of CircleCI.

Workflows, as this new capability in the updated CircleCI platform is called, was designed to help customers break up and map these processes according to their own needs.

Just released in private beta, CircleCI version 2.0 is essentially a rebuild of its build execution engine, undertaken in order to provide native Docker support. It provides the ability to spin up containers, put your build inside, then move that payload to whatever the next environment is. Workflows is the tooling that allows customers granular control to orchestrate those containers.

With the combination, it’s been able to get customer build times down by 60 to 80 percent, Rose said.

Rather than having to completely start builds over when tests fail, Workflows allows developers to designate exactly when, if, and how each job is run. It supports sequential and parallel jobs, as well as fan out, fan in, restart from fail and branch-level configurations.

Some people’s application pipelines are getting pretty convoluted, Rose said.

“You’re seeing instances where people have gone from a single monolith to literally hundreds of microservices and you end up pushing the challenge around testing and delivery away from unit testing and it becomes a problem of integration testing. So instead of testing the stability of one project, what you’re trying to test is the intersection of multiple projects together, and that can get pretty complicated,” he said.

The aim is increased speed.

In the move to microservices, tests that take 20, 30 or 40 minutes can be reduced to seconds or single minutes, he said. However, complexity comes from having way too many services. So the developer tries to find a happy medium to roll some of these services together so they test in discrete batches.

As a result, CircleCI needs to be able to support a range of approaches because customers are at every point in that spectrum, he said.

“Workflows is designed for that, to create that flexibility without being opinionated about the approach the customer takes. We’ve found that when we’re overly opinionated about the approach a customer should take, we usually get burned because everyone’s build, test, deploy process is unique. And it’s right for them,” he said.

CircleCI supports MacOS and any language that builds on Linux, including C++, Javascript, .NET, PHP, Python, and Ruby. Windows support is on its roadmap.

Version 2.0 also offers flexible resource allocation, expanded caching options to improve performance, custom build environments and the ability to debug via SSH or local builds.

“The North Star of what we’re trying to provide is productivity… trying to preserve developer minutes — spending less time on waiting for a build and more time on actually building things,” Rose said.

Its next focus will be for supporting projects using shared dependencies, shared libraries and open source as well as third-party API calls.

“There’s very little custom code used anymore. So you need to orchestrate those pieces to be able to deliver the application. That ultimately accelerates the development process because you’re not having to reinvent the wheel for things that are understood and been done before,” he said.

“The flip side of that, though, is that you no longer control your code. We run into customers all the time whose builds break not because of anything they did, but something changed inside their dependency graph,” Rose added. “Because hundreds of thousands of builds are done on the Circle platform, we can see these changes in the context of hundreds of builds at a time. We want to provide that insight to our customers so they can be more productive and worry less about the fragility of this dependency graph.”

Feature image via Pixabay.

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.