Building Open Source Projects Behind Company Firewalls – InApps Technology 2022

With the explosion of open source code sharing in recent years, engineers, developers, and maintainers have embraced this new way of having the community engaged to make the code better for all. Projects are built, software is written, documented and maintained, lives are made easier, the gods of coding are appeased until you get behind the firewall of a corporation.

Behind firewalls, the old guard holds the power. Practices long held to be true are strictly upheld, even if they are no longer working. But those who have seen the power of open source are beginning to change things. For the better.

Enter “innersource,” defined way back in 2000 by the Tim O’Reilly of O’Reilly Publishing, calling it “the use of open source development techniques within the corporation.”

Back in 2012, when innersourcing was beginning to gain traction, Red Hat expanded the definition:

“Properly implemented, the results can be dramatic – harnessing the energy and passion of your developers while helping drive software reuse and increased ROI in your organization…It also gives your developers freedom to show you what they can do when they are free to innovate.”

Innersource? So how does this work, exactly?

Speaking in a recent interview, Kakul Srivastava, vice president of product at GitHub,  said managing innersource boils down to two basic tenants: What’s the process for people coming in and out of this project?  And anyone on the project can find any information about any piece of the project.

“With the old model of enterprise, the focus is ‘we just want to get things working, get it done and get it out the door.’ Whereas with innersource you are thinking about ‘what will the experience be with other people you are interacting with during this project,’” said Brandon Keepers, GitHub’s open source lead, in the same interview.

He explained with an example of GitHub’s innersource process for opening up a new project. The process has a big checklist for opening a new project, including the “At Mention” feature where everyone on the team gets notified with a list of tasks to complete immediately when the project is open. He recently opened a project, filling in the description and proposed name. Within two minutes, someone from legal swung by to let him know that he did a search for trademarks and other conflicts, and this name looks clean. With that approval, the public relations team then launched into motion. In less than five minutes, logos were being created.

The Gifts Keep Coming

The efficiencies of innersource are the gifts that keep on giving. For example, at GitHub, all blog posts go through GitHub software. Someone writes a first draft, adds a pull request on the blog post repository. Other team members can come by and contribute, and the final draft triggers algorithms that check for culturally insensitive language and other internal criteria.

As the repository started filling up, patterns start to build up. The repository maintainer can observe patterns, then put rules in place that will run every time a new blog post is created. For example, the phrase “Today we are excited to announce…” goes stale in one day and will generate an automated message stating why the post was not approved.

All feedback is automated. “We have found that people are more receptive to feedback from pedantic robots than from pedantic humans,” said Keepers. “And robots tend to be a little bit more reliable.”

So why has the uptake been so slow? There are a couple of reasons, according to Srivastava. The first is that the GitHub product, used by most members of the open source community, is easily adapted to software development behind the firewall. GitHub has been using their software for internal projects for a long time. “GitHub has been doing innersource before innersource was cool,” said Srivastava.

The second is the huge conceptual shift is often hard for upper management to grasp. But with so many engineers using GitHub to manage their open source projects, it makes sense that they want to bring those efficiencies into their day jobs, sharing code with other developers.

When companies look to GitHub for innersource, they are looking for a software tool that helps their engineering teams, including open collaboration on code, open transparency between teams, company collaboration across divisions and departments and integration with automated testing.

Cultural Transformation Necessary

“What they’re really coming to us for a cultural transformation,” said Srivastava. “There are a lot of benefits to open source that companies may not realize. It’s cheaper, and the software is ‘alive’ hundreds of people working on the same project.”

Keepers said that many developers are already using open source for personal projects, or a younger hire comes on board who grew up used to working across the world, across time zones, across cultures and starts putting pieces into place. “Often times, by the time management is ready to embrace this, they find their developers are already well on the way,” he said.

He’s found executive management understands the concepts. “They are thinking a lot about how to do things faster better cheaper.” The challenge for GitHub is to train middle management, who are use to old ways of productivity, to embrace the ideas behind open source.

One of the beauties of running innersource is that you do not have to have any open source projects, said Srivastava. For example, large financial institutions have been embracing innersource source as a way of managing projects and contacting internally but are restricted to their use of open source by regulatory compliance.

So how does this work in the real world?

Bloomberg and PayPal

Take Panna Pavangadkar, Global Head of Engineering Developer Experience at Bloomberg, who fast-tracked innersource at Bloomberg. Taking up her position in late 2014 she started by listening to her engineers. It was obvious, she said, that the 4,000+ developers globally wanted more code sharing. Bloomberg’s corporate motto is “Be bold, take risks” so she decided to take the company at their word, she said in a recent phone interview.

Pavangadkar declared that all new projects would be run using innersource techniques. She figured she’d tell management about it later.

The Bloomberg Terminal, the company’s flagship product, has been around for about 30 years and was developed by a close team working with code sharing and other techniques we now see as open source. But in recent years, Bloomberg has expanded and servicing multiple sectors, are expanding and everything is not in a shared code base. This gave rise to the traditional approach, said Pavangadkar, where everything was siloed. Code was attributed to one person, and those persons became very possessive of their work.

Adopting Innersource

The 2015 InfoQ Innersource: Internal Open Source at PayPal, describes introducing open source practices into their engineering culture, and the results are exciting:

“The results were visible after six months. The Checkout Platform team spends 0% of its time rewriting code and just 10% reviewing submissions. The team was able to do a major refactoring and a 4x increase in performance without planning for it. The mindset moved from blocking change to mentoring and coaching.”

Breaking Down Silos

At Bloomberg, Pavangadkar said it took a lot of effort to break down silos from a pure entitlement perspective. By simply talking to her developers and listening to their concerns, she found a strong proprietary feeling toward their code and fear. Fear that their work wasn’t good enough for other people to see, and fear that some other developer would muck up their personal work.

She started with open up code repositories for viewing for all new projects, then developed rules of engagements to combat the developer’s concerns.

By creating a roadmap, every new engineer who is added to Bloomberg will be able to come up to speed on the culture and code of the Bloomberg Terminal and other Bloomberg products.

First rule: You’re not allowed to dish out dirt on any piece of code you see. Her two biggest messages: “Make sure your code does not introduce any risk” and “You’re not going to be judged.”

Not only did this combat these worries, but it also made the projects stronger internally.

Gradually, the developer’s sense of sole ownership shifted and they developed confidence in sharing their work.

Securing Success

Pavangadkar created a two-pronged approach. To senior management her message “Yes, this is worth exploring” and to development engineers, “Create new rules and get champions.”

Over the last 18 months, developers working with her have started coming to her saying, “Hey, I’m working on this project, and I can’t see the code base.” Pavangadkar explores the reasons for the silo. Sometimes it’s as trivial as getting passwords or security, sometimes more complicated, needing to educate managers in areas outside IT why code sharing is important.

An important part of the success of the project is documenting the process, Fleming said. “You don’t only put up code, you put up a document that says if you want to contribute, here’s how to do it,” he explained.  By creating a roadmap, every new engineer who is added to Bloomberg will be able to come up to speed on the culture and code of the Bloomberg Terminal and other Bloomberg products.

It’s important, said Fleming, to provide mentoring to new developers to maintain high quality.

But how does this go down with management? Pavangadkar brought a lot of qualitative metrics to sell innersource to management, bringing lessons learned from her projects.

Next up at Bloomberg

Build upon the best practices so that it becomes easier to collaborate, said Pavangadkar. Everyone has lessons learned, and as more projects come online, with the data they can define patterns, lessons learned, and which becomes beneficial for everybody.

Challenges around ownership, commitment, time spent are being addressed now, she said. Upcoming are plans to expand the Innersource mindset to sales and other non-tech staff so they understand how to use it and how it will benefit them.

Front Row Seat to Exciting Times

Back at GitHub, Keeper said, “This is all really exciting. We feel like we have a front row seat to all these amazing changes happening – every company is becoming a software company, and they are all starting to adopt open source and figuring out what that looks like. There are lots of exciting times ahead.”

Intrigued and want to find out how to get started? Check out this article on the Github wiki, or this one on how GitHub uses GitHub to document projects. Go forth and share.

Feature Image via Unsplash.