Disks Are Cheap, Your Time Is Expensive – InApps 2022

For the recording of our pancake breakfast at the OpenStack Summit Barcelona 2016, we invited CERN’s infrastructure manager,  Tim Bell, CERN distributed system architect Ricardo Rocha, as well as Red Hat Ansible community architect Robyn Bergeron, and Intel software developer Dan Bode to join us to chat about massive scalability, programmable infrastructure and the sometimes mind-staggering “Big Tent” complexity of OpenStack.

#119: OpenStack Summit Pancake Podcast – Disks Are Cheap, Your Time Is Expensive

Listen to all TNS podcasts on Simplecast.

What is programmable infrastructure? For Intel, the term means, “systems where you can write things as code or configuration, where you can describe your entire infrastructure in a revision control system, so you can do code review,” Bode said. This approach will give you a “built-in review process that you can use to understand what changes are being made, who are making the changes and why.”

When it comes to infrastructure, the CERN IT team relies heavily on automation. Their systems must collect the data that comes from particle-smashing Large Hadron Collider, which creates about a petabyte of data a second (which, thankfully, is filtered down to 2.5GB/sec to the organization’s two data centers, running about 190,000 cores).

The trick to scaling out to such massive builds is “scaling the number of machines without scaling the number of people to run them,” Bell said. Automation was the key here, he said.

Although CERN has traditionally used virtual machines to speed the deployment process for its researchers, it is now looking at containers to further expedite the workflow. To package a scientific application so it runs on CERN’s machines takes a lot of work, and researchers frequently need help with the task, Rocha noted. With containers running on OpenStack, “they will get an environment where they can focus on their work, instead of having to know the details of their infrastructure,”  Rocha said.

For this work, OpenStack’s Magnum comes in handy for CERN. “What we found attractive about Magnum as a framework was that we already got in place the OpenStack infrastructure,” said Bell, who is also an OpenStack board member. “So we had project frameworks, the workflows—all that extra tooling you have to invest in around the service. We could plug Magnum into that, and provide the users the flexibility of choosing their own orchestration engine.”

“CERN users are very technically advanced, and having a central team telling them they must use Kubernetes or they must use Mesos was not going to be an easy sell, whereas with Magnum, what we were able to do is tell people to make your own choice. It’s more about flexibility, compared to efficiency,” Bell said.

So not only can Kubernetes be used on top of OpenStack, but, with the aid of the OpenStack Kolla project, it can be used to deploy OpenStack itself. “Kolla is a project to deploy Ansible or deploy OpenStack in Docker containers,” Bergeron explained.

While the OpenStack Big Tent produces complexity, it’s that same complexity that paves the way for high scalability.

“I’m glad to see the amount of exploration people are doing, and I’m glad to see folks are better at doing open source stuff, and actually participating in all these communities, giving feedback. It actually means [this] might all be viable,” Bergeron explained.

This pancake podcast, and this post, has been sponsored by Intel:

Intel’s Matthew Brender Discusses Telemetry and the Intel Snap Tool

Listen to all TNS podcasts on Simplecast.

Intel and Red Hat are sponsors of InApps.

Feature image: (left to right): Ricardo Rocha, Robyn Bergeron, Tim Bell, Dan Bode.

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.