The Switch to Open Source is Working for .NET and Microsoft – InApps 2022

Microsoft had a clear strategy when it open sourced .NET. How is it playing out? Making .NET open source lets Microsoft build it faster, get it onto more platforms (you can run the common language runtime [CLR] on OS X now) and take advantage of community contributions. It’s also changing the way .NET is developed.

Some of that is the nature of open source. For one thing, taking contributions from the community and implementing pull requests means that contributors scratch their own personal itches, down to improving comments or fixing the formatting of the code.

Seemingly trivial changes in Microsoft open source projects don’t get rejected. The mind-set is changing from the view that a product team is letting people play with its toys, to an appreciation of the fact that the more you let the community have an impact, the more committed they’ll be to the product.

And when it comes to internationalization issues, something as simple as a comma can be the problem.

Getting that community involvement is a priority, because it’s what will make .NET truly cross-platform. As Immo Landwerth of the .NET team puts it, “if you want to do cross-platform for real, you kind of have to be an open source stack; look at all the other cross-platform tools out there, like Linux and browsers. If you move it out in the open, the people who care about a certain architecture will jump in.” And they did.

Of the first 100 pull requests for .NET, more than 40 came from the community, according to Dave Kean of the .NET team. Many of those were small changes, but there have already been more substantial contributions. By the end of January, there were too many forks for GitHub to display as a graph: over 1,000. By that time, there had been nearly 250 pull requests and 51% of those came from the community.

In fact, it looks like Microsoft was surprised by the amount of participation and contribution, especially for OS X. CoreCLR initially had support for Windows and Ubuntu, with OS X support planned for later, after the team had done some more preparation. But so many people wanted to work on it — including Geoff Norton from the original Mono team who had worked on Mono for OS X and contributed the code to get CoreCLR onto OS X — that Microsoft quickly created a feature branch and migrated the work it had already done on that preparation to GitHub so the community could join in.

In practice, that discussion and collaboration took a few days to resolve, with some frustration along the way. Microsoft engineers have a lot more experience running internal projects than they do running high-profile open source projects, and they’re still getting up to speed with the process. But it did get resolved, and Microsoft did the right thing, so it’s clear that they’re committed to this and want to do the right thing, even if they don’t always move as fast as the community.

“We’re still learning,” says Landwerth repeatedly.

They’ve recently highlighted a number of areas as ‘up for grabs’ to make it clear what they’re not working on (and you can see what they are working on as it moves into public feature branches). The new .NET Foundation Advisory Council (http://www.dotnetfoundation.org/blog/welcoming-the-newly-minted-advisory-net-foundation-advisory-council-members) isn’t a rubber-stamp list of Microsoft staff and existing .NET projects either; there are some familiar, credible, open source names there, like Debian and GitHub.

Just putting .NET onto GitHub meant some fairly substantial changes in the way the team works. Their internal engineering systems were all based on Microsoft’s Team Foundation Server, which is great for collaborating with people inside Microsoft, using Visual Studio, but it wasn’t going to work for everyone else.

“To bring .NET into the open source, we needed the engineering system to be open source,” Kean points out. “We don’t want to have snapshots of the source code that we give you every once in a while. We need to give you the real deal.” That meant changing the way the internal systems work by adding a two-way mirror to GitHub so the code could be live on both systems. To prepare the code, Microsoft also used Roslyn to write an automatic code formatter to convert it all into the .NET-approved coding style (older, core libraries often had different coding styles).

That’s also why the codebase has been going up piece by piece: first the out-of-band libraries, like mutable collections, and the metadata reader used for cross assemblies and xml; then bigger pieces like primitives, the registry, the file system and the CoreCLR execution engine — which even includes the garbage collector. (The garbage collector is big in more than one sense. It’s too large a file for GitHub to pretty print; at one point Microsoft’s build system had to be rewritten to handle the file size. It was originally written in LISP, then machine translated to C; and one of the Microsoft Technical Fellows has been working on it for years, adding features.)

A bigger change is moving internal processes into the community. If the .NET Core code that’s getting a code review is on GitHub, the code review now happens on GitHub — even if it’s code from an internal Microsoft developer. The idea is that “there’s really no difference between the external and the internal contributor; both will submit a pull request, both will get feedback from Microsoft people as well as the community, and then it gets decided whether things are in or not in,” says Landwerth. “This really blurs the line; it doesn’t matter who you are.”

In time, internal and external pull requests will run through the same internal code, checking for things like internationalization and compatibility. That might slow things down, because the compatibility testing can take several days, but it paves the way for major contributions from the community because they can be treated the same way internal development is.

The .NET team does regular API reviews (several times a week if there are enough requests and changes to deal with). Open source contributors can’t go to those meetings, but there’s a process for community discussion first, and video recordings and notes from the API go onto Microsoft’s Channel 9 site and a GitHub wiki, so everyone can see how decisions were made.

That’s part of how the .NET team wants to get Microsoft’s “tribal knowledge” out to the community.

“Internally, there is a site where we have every single discussion they had in [the] C# and .NET [teams] since the very beginning,” Landwerth explains. “There is so much insight in there, into why you did certain things, why you rejected things.” He’s also enthusiastic about changing the process, and, in general, having as little process as possible. So if the current system for doing API reviews doesn’t work for the community, you can expect it to change.

.NET is far from the only serious open source project at Microsoft: there’s the Roslyn compiler, F#, the Azure SDKs, TypeScript, TouchDevelop, Project Orleans, Bond… At the beginning of February there were 840 Microsoft repos on GitHub (there may be even more by now).

And all this is starting to change some other things inside Microsoft. CEO Satya Nadella has repeatedly talked about “internal open source” — getting different divisions to share their code with each other rather than reinventing the wheel every time — that’s started internally, through internal systems. The source code for Visual Studio Online (VSO) is available internally, and the Office team has contributed code they use for building mobile applications to the next version of the VSO build system. But some Microsoft developers are also getting code from Microsoft teams in other divisions — by getting it from their GitHub repo.

Development in the open isn’t for everyone, and it doesn’t work for all projects. But it clearly has benefits, even in as traditional a development organization as Microsoft.

Featured image via Flickr Creative Commons.