How Microsoft Is Shifting Focus to Open Source – InApps Technology 2022

Open source is becoming the de facto software development model. Today every software company, especially in the enterprise space, is dealing with a massive amount of open source. And Microsoft is no exception.

In fact, Microsoft seems to be accelerating in this work, catching up with rivals more traditionally associated with the open source ecosystem. The company has the largest number of the top 500 open source projects for any one entity, according to a compilation from the Cloud Native Computing Foundation. Microsoft has 24 projects, way ahead of Google and Pivotal (each having seven) and Red Hat, which has six.

With its own open source projects like .Net Core, Visual Studio Code and TypeScript, Microsoft is getting considerable help from the outside open source community. Over 60 percent of the contributions in CoreFX .Net core libraries and CoreCLR .Net Core runtime are coming from outside Microsoft. And the company is contributing significantly to external outside open source projects, such as Linux, Docker, and Kubernetes.

New Culture

The first open source from Microsoft arrived back in 2004 when the company released a Windows development toolset called WiX, which came with a compiler, a linker, a lib tool and a decompiler. The company has been releasing open source ever since.

In 2012, Microsoft became the largest contributor to the Linux kernel. Ky Srinivasan, who is now a Microsoft software architect for the Windows Server Division, was the engineer driving many of the contributions, most concerned facilitating Hyper-V support for Linux. “Since then I’ve been trying to make Linux run as well, if not better, than any other platform that’s out there,” he said.

Srinivasan’s work also made him the top contributor to Linux, in the volume. His second priority was to get this code into the mainline tree so that other distros could ship Linux support to Hyper-V out of the box. Today, all major Linux vendors are doing it.

With this work, Microsoft truly saw first-hand the benefits of open source. Srinivasan said that Linux has been so successful on Microsoft platform that Microsoft’s contribution to Hyper-V related patches has fallen to a little less than 60 percent. The rest of the patches are coming from the community, many from Linux distributors such as SUSE, Red Hat and Canonical.

On the Azure side, Microsoft is working with all major commercial Linux vendors — including Red Hat, SUSE, Canonical, CoreOS, etc. — and other communities to ensure that customers can run their preferred Linux workloads on Azure.

For developers, Microsoft is now bringing Linux utilities to Windows natively. Microsoft has been working closely with Canonical to build Windows Subsystem for Linux (WSL) that enables developers, sysadmins and operations teams to use all the *NIX capabilities natively on Windows allowing them to manage their Linux workloads running on Azure or any other cloud.

Mark Shuttleworth, the CEO of Linux distributor Canonical, thinks it’s a really good thing: “I think that will mean more users. More bug reports. More engagement. More eyeballs. More participation. More usage. I think that’s all very good.”

The Road Ahead

As Microsoft weaves its way through the Linux world, it has some serious challenges ahead of it. Linux was not designed to run on Windows, and Windows was not designed to host Linux. That’s where the “working together” part comes into play.

Feature Image: Docker Technical Staff Patrick Chanezon, flanked by Microsoft Azure Architect John Gossman (Left) and Microsoft principal programming manager Taylor Brown, at Dockercon EU 2015.

“We are trying to bring together these two different worlds, and the whole time we’re able to really make them work well together. And that’s a technical challenge that we’ve pretty more or less solved and understand what to do there because both sides are now willing to talk. They’re able to influence the design of Windows going forward to better support Linux, and we’re also able to talk to the Linux community and make sure that things are done in a way that doesn’t seclude running Windows,”  Srinivasan said.

Microsoft has also been working with Docker to create special editions of Docker for Windows and Azure. The work the two companies did together for the last couple of years resulted in another open source project called LinuxKit.

LinuxKit was based on the work that Microsoft did on the boot2docker project, noted Taylor Brown, lead program manager for Windows Server and Hyper-V Containers. Microsoft also worked on bringing Linux containers to Windows Servers by adding support for Hyper-V isolation to Linux containers running on Windows. It means that over 900,000 Docker containers hosted on DockerHub can run on Windows servers, bringing those containers to users.

This year, Microsoft also acquired Deis to bring the container expertise in-house. With this acquisition, Microsoft has also become a major contributor to the Kubernetes ecosystem with projects like Workflow, Helm, and Steward.

Helping the Ecosystem

Open Source is all about community and ecosystem. No company can survive without a healthy ecosystem and Microsoft knows it very well. The company has been investing heavily in open source communities. It joined the Linux Foundation and John Gossman, the lead architect Microsoft Azure, became a member of the board. The company recently joined the Cloud Foundry Foundation as a top tier member as well.

The company has over 15,000 contributors on GitHub. The company said that over 6,000 employees contribute to open source projects, and have released over 3,000 open source projects. Microsoft’s open source programs office tracks nearly 10,000 open source components, everything from NPM packages to Linux distros used by Microsoft teams.

The company recently wrote an operating system for Azure Networking switch called SONiC, which is powered by Linux. Yes, Microsoft is using Linux in its own infrastructure. The company also recently moved Windows development to Git.

SONiC is not an isolated case of open source consumption by the company. Microsoft is a heavy consumer of open source. “We have a list of over 10,000 different open-source components that are being used and I’m actually one of the approvers for that list,” said Gossman.

Here are some of the open source technologies that Microsoft uses and maintains internally:

• Kafka: LinkedIn and Bing are two of the biggest users of Kafka.
• Linux: About one in three VMs on Azure are running Linux.
• LightGBM and the Microsoft Cognitive Toolkit (CNTK) machine learning frameworks.
• Linux-based Hadoop and Spark services on Azure.
• Postgres and MySQL managed services on Azure.
• Azure Functions, an open source serverless Functions-as-a-service.
• Chef is used to manage thousands of nodes internally across Azure, Office 365 and Bing.

The company has invested heavily in hiring the best open source talent. Microsoft is now home to some of the founders and top developers of open source technologies including:

• Brendan Burns: Burns was a co-founder of Kubernetes, and is now an architect at Microsoft, leading the team contributing to upstream Kubernetes.
• Gabe Monroy: Monroy created Deis, and is now lead program manager on containers at Microsoft.
• John Howard: Howard is software engineer with Azure and top contributor to Docker over past 12 months.
• Anders Hejlsberg: Creator of C# and Turbo Pascal, Hejlsberg is a technical fellow leading the development of Typescript.
• Erich Gamma: A distinguished engineer who is one of the fathers of the Eclipse project and “Gang-of-Four” authors of software engineering textbook, “Design Patterns: Elements of Reusable Object-Oriented Software,”  Gamma is leading the team on Visual Studio Code.
• Ross Gardler: Former president of the Apache Software Foundation, Gardler is now a program manager working on Azure Container Service and Open Source.
• Maggie Pint: Pint is a software engineer lead for in Azure Site Reliability Engineering (SRE) and maintainer of a popular open source library called Moment.js, and a delegate of the JavaScript Foundation to TC39.

Cultural Shock

All of this work means a major shift in culture within Microsoft, which has long been a proprietary company. Srinivasan jokingly recalled that when he joined Microsoft in 2011, being a hardcore Linux engineer, not too many people would talk to him and he was constantly reverse engineering a lot of things with Windows in mind.

But things have changed over time. “Now I participate in pretty much all the all the design discussions with the Windows kernel team, and there’s a very close relationship now between folks in my team and the Windows kernel development architecture team,” he said.

Michelle Noorali, a software engineer who came to Microsoft through Deis acquisition shared the same story. She is now a senior software engineer for Microsoft Azure. Noorali had never worked at a big company before, she has worked only with start-ups so it was obvious that she was very hesitant when she moved to Microsoft. “I was very cautious about going forward, but honestly I am so impressed. People really appreciate what we’re doing, what the Deis team is doing in the open-source community and want that to be a part of the culture and have welcomed it with open arms,” she said.

Srinivasan attributed this change to the top leadership of Microsoft.

“The culture within Microsoft is changing as we speak. Linux expertise is spreading across the company. As you know, there are a number of embedded Linux deployments within Microsoft, and so it’s a much more Linux friendly place and each day it’s friendlier than the previous day. It’s really nice now,” said Srinivasan.

Microsoft will be discussing all matters open source this Wednesday, June 21 at the Azure OpenDev virtual conference. 

InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.

Feature image via Pixabay.