A Cloud Native Testing Framework for Kubernetes – InApps Technology 2022

The CNCF Cloud Native Landscape is filled with hundreds of open source projects and open-core companies estimated to be worth $15.6 trillion, according to the Cloud Native Computing Foundation.

And likely worth even more to the businesses, from small development companies to multinational enterprises, that rely on cloud native technology every day.

The landscape is made up of well-understood categories, like databases, configuration management, security, monitoring and machine learning, each with a handful of projects that have ultimately changed the way organizations create, deploy and maintain applications in the cloud. Think Kubernetes, Helm, Redis, Grafana, Ansible and more.

Perhaps somewhat surprisingly, to us at least, there is no dedicated category for testing/QA in the CNCF landscape. For now, testing has been assigned to the Continuous Integration and Delivery category.

That’s how we got started with Testkube — reflecting on our own testing pains with Kubernetes and trying to understand precisely where and why our past deployments failed using industry-standard tools like Postman and Cypress. We got to work building a framework that natively integrates test orchestration and execution into Kubernetes and your CI/CD/GitOps pipeline.

Today, we’re shipping version 1.0 of this cloud native testing framework, which we believe to be the first in the ecosystem. We would like to think we’re taking the first step in defining a new testing category in the CNCF landscape, but time will tell. Our challenge is clear.

What’s the Testkube Vision for Kubernetes Testing?

We built Testkube because we believe that cloud native software applications require not only new tools for building, deploying or monitoring, but testing as well. Popular testing tools of today such as Postman, Cypress or K6 were not specifically designed for Kubernetes. We thought, “How could we bridge that gap?”

Unchain Test Orchestration and Execution

We believe that testing should not be coupled with CI/CD.

In many teams, automated testing is tightly coupled to their CI/CD workflow, which makes it harder to change and experiment with new testing tools without making some changes to the CI/CD pipelines. With Testkube, you’re no longer chained to your CI/CD tool when picking how you want to run tests for your applications in staging or production environments. Testkube simplifies test management, orchestration and execution by allowing tests to be stored, executed and managed as part of a cluster through the Testkube CLI or its API service.

Bring Your Own Application Testing Tools

We believe you should be able to use any testing tools on your Kubernetes cluster.

You should be able to run functional, load/performance, security and compliance tests from within your cluster without having to wrap tests in complex Docker containers, grant risky network access to your cluster or rely on external services for test execution.

Testkube provides a modular test-execution framework with out-of-the-box executors for popular testing tools like Postman, Cypress.io or K6, allowing teams to use existing testing assets. You can also create your own executors for any kind of tests you would like to run from within your cluster.

A Single ‘Quality Control Plane’

We believe that you should have a holistic view of all test results in your clusters.

We suggest better workflows than constantly running kubectl and returning logs to understand how tests are performing, debug issues and pinpoint failures. This should happen on a single dashboard, which returns not only results but also logs and artifacts for actionable QA analytics. Testkube 1.0 is taking a step to make this vision a reality.

The included dashboard provides you with a “warning light” for your Kubernetes cluster, much like you do for your car, that tells you something is wrong and stores all the necessary context for diagnosing and troubleshooting an issue right in the cluster itself.

Your Centralized Kubernetes Dashboard for Testing

Testkube’s UI lets you see all tests and test suites at a glance, along with details on whether they’ve passed, are currently running or are scheduled to run.

On the dashboard, you can quickly see exactly which test suites have failed. You’re two clicks away from understanding which specific executions have failed, when they started, their run time and the relevant steps you might want to investigate further.

Now that you know both your Postman or Cypress tests have failed, you can explore individual executions to view the results, log output and any applicable artifacts. You have all the right resources to get back to fixing bugs rather than running kubectl yet again or hopping between multiple browser tabs to compare logs with artifacts.

We also added labeling functionality to help organize your test suites and individual tests in meaningful ways. Because Kubernetes has permissive labels, you can enrich Testkube with the same labels you might have already applied to your Kubernetes resources and Kubernetes nodes — think labels like backend, frontend, staging, production and more — to narrow and help focus your testing efforts.

A More Familiar Testkube CLI Experience

You’re probably familiar with the syntax for kubectl at this point: kubectl + command/verb + resource type + name. You can’t get very far in the Kubernetes world without that becoming part of your muscle memory. We’ve overhauled Testkube’s CLI experience to match what you’re already familiar with.

For example, testkube get test [test-name] now returns information about a specific test.

It might seem like a small change, but we think that sharing a syntax will help developers get familiar with Testkube a lot faster, and we’re all about improving the developer experience around cloud native testing in general.

Bring Your Own Kubernetes Testing Tools

The last new feature of Testkube v1.0 is the possibility to integrate any testing tools/platforms with the Testkube execution engine through executors.

Testkube includes executors for Cypress for UI testing and Postman for improving the resiliency of your backend applications, and we’re excited to ship first-class support for k6, an open source load testing tool.

K6 lets developers write tests, using APIs and Javascript, that are flexible and mimic real-world scenarios that strain a Kubernetes cluster, including CPU and memory usage, disk usage on persistent volumes and overall resource usage. It’s a great tool for examining your cluster’s reliability and performance in trying situations and at scale, combining easy onboarding with tons of available complexity.

The First Big Community Win

The k6 executor was developed and contributed to Testkube by an external contributor, Leander Reimer, which is an exciting development for a young project like ours. He happened to be working on this executor independently when our team announced we wanted to support k6 to bring load testing to Kubernetes clusters. We were thrilled to partner to make the code launch-ready.

If you’re interested in building your own executor to extend your cloud native testing capabilities, we’ve designed the codebase to be entirely modular. You don’t need a deep understanding of Testkube as a whole. We’ve put together several resources to help you get started with contributing, including a document on creating a custom executor and a template executor repository for those who want to build off a reliable framework. The template uses Go, but you can write an executor in any other programming language, like Rust, Javascript, Clojure and more.

Looking to the Future of Kubernetes Testing

While Testkube v1.0 is a major step, we have big ambitions. Our next efforts will extend the dashboard’s functionality, such as providing robust reporting on your testing over time or through relevant cross-sections to help identify growth areas in your organization’s development life cycle.

Our support for Cypress, Postman and k6 is just a beginning. We’re already working with Reimer on executors for Maven and Gradle, and we are scoping out the work to add support for SoapUI, Selenium, Cucumber and others.

And once you’ve had time to install Testkube and define your first tests, I’d love to hear from you about your experiences. Book a time with me and we’ll figure out how to make Testkube better for you today. Or join our Discord! As we start looking ahead to v2.0, now is the perfect time to help shape the future of cloud native testing.

InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker, Postman.

Photo by Sigmund on Unsplash