Posts

open source program

“We believe that our projects help move the industry forward while giving other companies and individuals the opportunity to use our platform to scale more quickly and build better products.” – Christine Abernathy, Open Source Developer Advocate at Facebook.

Facebook’s open source team was “formally” created in 2009, but the company has built with open source from its inception. Facebook.com was originally built on top of the LAMP (Linux/ Apache/ MySQL/ PHP) stack. And over time Facebook has used and contributed back to these projects, as well as evolved and released new projects such as Hack which has its roots in PHP.

“Open source is core to our engineering DNA. We believe that sharing our code and even hardware designs accelerates the pace of innovation in the world. We believe that our projects help move the industry forward while giving other companies and individuals the opportunity to use our platform to scale more quickly and build better products.” – Christine Abernathy, Open Source Developer Advocate at Facebook.

Custom tools to manage open source

Facebook has a dedicated Tools team within the open source program office that is responsible for building internal tools to help manage its open source portfolio. This includes the projects that Facebook shares, which are mostly hosted on GitHub, as well as the other external projects they contribute to such as the Linux Kernel.

The program office provides a dashboard for each project that includes GitHub metrics such as the number of open issues or the ratio of internal to external contributions for a given time period. Project maintainers are given tools so they can bring GitHub pull requests and issues into their internal review and bug tracking systems. This makes it easier for engineers to manage external issues where they’re most comfortable. Maintainers also have access to workflow tools to reliably push internal commits out to GitHub, making it easy to quickly sync internal and external code bases and reducing the churn on landing external contributions.

The open source team can look at these project dashboards to help analyze the health of a particular project. They’ve even open-sourced some of these workflow tools: mention-bot and FBShipIt.

The team collects top-level statistics on how the overall portfolio is doing through aggregate dashboards across the GitHub orgs they manage. These are used to provide high-level reports to stakeholders and a community of internal open source enthusiasts. The tools team also provides insight into top contributors. Project maintainers are encouraged to refer to this list and reward their top contributors. The company periodically thanks its top internal contributors and makes some of this information available to its internal review systems.

The open source office also provides tooling to guide potential projects through the review process. This helps streamline the process and helps the team easily spot and correct bottlenecks.

The open source team also provides services such as documentation. This includes helping out with the technical content as well as building out some of the documentation infrastructure and templates that projects can use.

Open Source Success Through Steady Progress

At the end of each half the program office identifies goals around metrics they want to achieve. The metrics they track include:

  • The average age of open issues or open pull requests
  • The ratio of external to internal commits
  • The number of commits
  • The growth in followers and forks
  • The number of social media followers.

They’re periodically tweaking what they measure as they refine what it means to maintain a healthy portfolio.

Facebook also surveys new hires every six months to gauge their awareness of its open source program. They set baseline metrics a few surveys back and the goal is to maintain or grow those numbers.

Their open source success isn’t the result of one action but the cumulative effect of a steady stream of quality releases over the years and a focus on growing thriving communities to support those projects.

“Projects like React, React Native, Create-React-App, Immutable, HHVM, Fresco, and GraphQL are the constant beat that have contributed to the success of our program,” Abernathy said.

One of Facebook’s most successful projects is React Native. It makes use of many of Facebook’s tools to help manage the community. For example, mention-bot came out of this project and was a way to quickly identify reviewers for a pull request. FBShipIt helped it cut down the time to bring in external contributions, review them internally, and land these contributions back out to GitHub. In the early days, this process sometimes took a day as much of it is manual. Now this can be done in as little as minutes if it’s an automatic reviews.

The open source program office also provided documentation services to help refresh and keep the React Native site up to date.

Tips for New Open Source Program Managers

Organizations that are just establishing an open source strategy and program office can learn from the success of Facebook’s open source program. Here are the three key practices that Abernathy shared that have contributed to their success as a program office:

  1. When evaluating what to share, it should be something that’s useful to your company. Many of the projects that Facebook shares are used in production and include all the benefits that come along with that. This means those projects are likely to have continued support which in turn means the community is well supported.
  2. Find a way to highlight, promote, and reward your open source contributors, both internal and external. Facebook has periodic reports that highlight its open source heroes. This helps raise the profile of engineers and their work with managers who may sometimes not be managers of that open source project.
  3. As a central program office, find pain points that cut across the various projects and tackle them.

For example, many projects had previously built their own commit copying scripts and it was the number one pain point from a survey they ran at that time. FBShipIt, which copies commits between repositories, was built to address this and it’s owned by the open source team. It moved the burden off the engineering teams and is universally praised for helping smooth the workflow for pulling in external contributions.

Acknowledgments

For this feature we interviewed Christine Abernathy (@abernathyca), Open Source Developer Advocate at Facebook, to learn more about the Facebook’s open source program. Libby Clark performed the interview.

The 2017 Linux Kernel Report illustrates the kernel development process and highlights the work of some of the dedicated developers creating the largest collaborative project in the history of computing.

Roughly 15,600 developers from more than 1,400 companies have contributed to the Linux kernel since 2005, when the adoption of Git made detailed tracking possible, according to the 2017 Linux Kernel Development Report released at the Linux Kernel Summit in Prague.

This report — co-authored by Jonathan Corbet, Linux kernel developer and editor of LWN.net, and Greg Kroah-Hartman, Linux kernel maintainer and Linux Foundation fellow — illustrates the kernel development process and highlights the work of some of the dedicated developers who are creating the largest collaborative project in the history of computing.

Jens Axboe, Linux block maintainer and software engineer at Facebook, contributes to the kernel because he enjoys the work. “It’s challenging and fun, plus there’s a personal gratification knowing that your code is running on billions of devices,” he said.

The 2017 report covers development work completed through Linux kernel 4.13, with an emphasis on releases 4.8 to 4.13. During this reporting period, an average of 8.5 changes per hour were accepted into the kernel; this is a significant increase from the 7.8 changes seen in the previous report.

Here are other highlights from the report:

  • Since the last report, more than 4,300 developers from over 500 companies have contributed to the kernel.
  • 1,670 of these developers contributed for the first time — about a third of contributors.
  • The most popular area for new developers to make their first patch is the “staging tree,” which is a place for device drivers that are not yet ready for inclusion in the kernel proper.
  • The top 10 organizations sponsoring Linux kernel development since the last report are Intel, Red Hat, Linaro, IBM, Samsung, SUSE, Google, AMD, Renesas, and Mellanox.

Kernel developer Julia Lawall, Senior Researcher at Inria, works on the Coccinelle tool that’s used to find bugs in the Linux kernel. She contributes to the kernel for many reasons, including “the potential impact, the challenge of understanding a huge code base of low-level code, and the chance to interact with a community with a very high level of technical skill.”

You can learn more about the Linux kernel development process and read more developer profiles in the full report. Download the 2017 Linux Kernel Development Report now.

APIs

Learn tricks, shortcuts, and key lessons learned in creating a Developer Experience team, at APIStrat.

Many companies that provide an API also include SDKs. At SendGrid, such SDKs send several billions of emails monthly through SendGrid’s Web API. Recently, SendGrid re-built their seven open source SDKs (Python, PHP, C#, Ruby, Node.js, Java, and Go) to support 233 API endpoints, a process which I’ll describe in my upcoming talk at APIStrat in Portland.

Fortunately, when we started this undertaking, Matt Bernier had just launched our Developer Experience team, covering our open source documentation and libraries. I joined the team as the first Developer Experience Engineer, with a charter to manage the open source libraries in order to ensure a fast and painless integration with every API SendGrid produces.

Our first task on the Developer Engineering side was to update all of the core SendGrid SDKs, across all seven programming languages, to support the newly released third version of the SendGrid Web API and its hundreds of endpoints. At the time, our SDKs only supported the email sending endpoint for version 2 of the API, so this was a major task for one person. Based on our velocity, we calculated that it would take about 8 years to hand code every single endpoint into each library.

This effort involved automated integration test creation and execution with a Swagger/OAI powered mock API server, documentation, code, examples, CLAs, backlogs, and sending out swag. Along the way, we also gained some insights on what should not be automated — like HTTP clients.

In my talk at APIStrat, I am going to share some tricks, automations, shortcuts, and key lessons that I learned on our journey to creating a Developer Experience team:

  • We will walk through what we automated and why, including how we leveraged OpenAPI and StopLight.io to automate SDK documentation, code, examples, and tests.
  • Then we’ll dive into how we used CLA-Assistant.io to automate CLA signing and management along with Kotis’ API to automate sending and managing swag for our contributors.
  • We’ll explore how these changes were received by our community, how we adapted to their feedback and prioritized with the RICE framework.

If you’re interested in attending, please take a moment to register and sign up for my talk. I hope to see you there!

OpenStack Summit Sydney

OpenStack Summit Sydney offers 11+ session tracks and plenty of educational workshops, tutorials, panels. Start planning your schedule now.

Going to OpenStack Summit Sydney? While you’re there, be sure stop by The Linux Foundation training booth for fun giveaways and a chance to win a Raspberry Pi kit. The drawing for prizes will take place 1 week after the conference on November 15.

Giveaways include The Linux Foundation projects’ stickers, and free ebooks: The SysAdmin’s Essential Guide to Linux Workstation Security, Practical GPL Compliance, A Guide to Understanding OPNFV & NFV, and the Open Source Guide Volume 1.

With 11+ session tracks to choose from, and plenty of educational workshops, tutorials, panels — start planning your schedule at OpenStack Summit in Sydney now.

Session tracks include:

  • Architecture & Operations
  • Birds of a Feather
  • Cloud & OpenStack 101
  • Community & Leadership
  • Containers & Cloud-Native Apps
  • Contribution & Upstream Development
  • Enterprise
  • Forum
  • Government
  • Hands-on Workshop
  • Open Source Days
  • And More.

View the full OpenStack Summit Sydney schedule here.

Cloud Native Computing Foundation and Cloud Foundry will also have a booth at OpenStack Summit Sydney. Get your pass to OpenStack and stop by to learn more!

“Recruiting Open Source Developers” is a free online guide to help organizations looking to attract new developers or build internal talent.

Experienced open source developers are in short supply. To attract top talent, companies often have to do more than hire a recruiter or place an ad on a popular job site. However, if you are running an open source program at your organization, the program itself can be leveraged as a very effective recruiting tool. That is precisely where the new, free online guide Recruiting Open Source Developers comes in. It can help any organization in recruiting developers, or building internal talent, through nurturing an open source culture, contributing to open source communities, and showcasing the utility of new open source projects.

Why does your organization need a recruiting strategy? One reason is that the growing shortage of skilled developers is well documented. According to a recent Cloud Foundry report, there are a quarter-million job openings for software developers in the U.S. alone and half a million unfilled jobs that require tech skills. They’re also forecasting the number of unfillable developer jobs to reach one million within the next decade.

Appeal to motivation

That’s a problem, but there are solutions. Effective recruitment appeals to developer motivation. If you understand what attracts developers to work for you, and on your open source projects (and open source, in general) you can structure your recruitment strategies in a way that appeals to them. As the Recruiting Open Source developers guide notes, developers want three things: rewards, respect and purpose.

The guide explains that your recruitment strategy can benefit greatly if you initially hire people who are leaders in open source. “Domain expertise and leadership in open source can sometimes take quite a long time at established companies,” said Guy Martin, Director of Open at Autodesk. “You need to put training together and start working with people in the company to begin to groom them for that kind of leadership. But, sometimes initially you’ve got to bootstrap by hiring people who are already leaders in those communities.”

Train internal talent

Another key strategy that the guide covers is training internal talent to advance open source projects and communities. “You will want to spend time training developers who show an interest or eagerness in contributing to open source,” the guide notes. “It pays to cultivate this next level of developers and include them in the open source decision-making process. Developers gain respect and recognition through their technical contributions to open source projects and their leadership in open source communities.”

In addition, it makes a lot of sense to set up internal systems for tracking the value of contributions to open source. The goal is to foster pride in contributions and emphasize that your organization cares about open source.  “You can’t throw a stone more than five feet in the cloud and not hit something that’s in open source,” said Guy Martin. “We absolutely have to have open source talent in the company to drive what we’re trying to do moving forward.”

Startups, including those in stealth mode, can apply these strategies as well. They can have developers work on public open source projects to establish their influence and showcase it for possible incoming talent. Developers have choices in open source, so the goal is to make your organization attractive for the talent to apply.

Within the guide, Ibrahim Haddid (@IbrahimAtLinux) recommends the following strategies for advancing recruitment strategies:

  1. Hire key developers and maintainers from the open source projects that are important to you.
  2. Allow your developers working on products to spend a certain % of their time contributing upstream.
  3. Set up a mentorship program where senior and more experienced developers guide junior, less experienced ones.
  4. Develop and offer both technical and open source methodology training to your developers.
  5. Participate in open source events. Send your developers and support them in presenting their work.
  6. Provide proper IT infrastructure that will allow your developers to communicate and work with the global open source community without any challenges.
  7. Set up an internal system to track the contributions of your developers and measure their impact.
  8. Internally, plan on contributing and focus on areas that are useful to more than one business unit/ product line.

The Recruiting Open Source Developers guide can help you with all these strategies and more, and it explores how to weave open source itself into your strategies. It is one of a new collection of free guides from The Linux Foundation and The TODO Group that are all extremely valuable for any organization running an open source program. The guides are available now to help you run an open source program office where open source is supported, shared, and leveraged. With such an office, organizations can establish and execute on their open source strategies efficiently, with clear terms.

These guides were produced based on expertise from open source leaders. Check out the guides and stay tuned for our continuing coverage.

Also, don’t miss the previous articles in the series: How to Create an Open Source Program; Tools for Managing Open Source Programs; and Measuring Your Open Source Program’s Success.

Sunday I will partake in something I’ve never tried before: a Tough Mudder (TM).  If you’re not familiar with this event, it’s a 10-12 mile run with an obstacle about every half mile. Most people participate with a team, and it’s generally not timed.  It looks something like this:

(Electroshock Therapy – all pics from http://toughmudder.com)

I’ll admit I’m a little nervous. Will I be able to make it through the obstacles? What happens if I wear out and let my teammates down?

But I think I have a secret weapon. It’s not miles of running or hundreds of pull-ups and pushups. It’s not having the right gear…

It’s participating in open source.

Yep, I think that having participated in open source – specifically in the Open Network Automation Platform (ONAP) project where I’m on the TSC — will be the differentiator in helping me get through the Tough Mudder.  It all comes down applying what I’ve learned in open source to the obstacles along the way…

Lesson #1: You’re going to get dirty

Open source is not always a clean process.  It doesn’t run top-down like software development inside of most companies. There will be things that go wrong. People will have differences of opinion. You have to expect it and find a way to work through it. In ONAP we had to merge two different projects (OpenECOMP and Open-O) into our first release due in November. There was overlap between the projects, and (naturally) pride of ownership.  But the community made the tough decisions and did the hard work to bring the best of each project into the final product.

Knowing I’m going to get dirty, I’m now prepared for the Mud Mile…

Lesson #2:  It can be intimidating

If you’ve never worked in open source before, getting involved can be intimidating. Everyone seems so smart, what can I possibly contribute?  I’ve learned that you can start small with documentation, testing, or easy bug fixes, and as you build up your reputation, you can take on more responsibility.

Knowing that it can be intimidating, I’m now ready for the King of the Swingers.

Lesson #3: You can do things that you’ve never imagined

The great thing about open source is that it’s often tackling new and interesting problems. Many of the best new technologies today are coming from open source, and if you participate, you can say that you helped build it. In ONAP, we’re working on making the network virtualized & running it at a scale never seen, while still keeping the 100-year-old expectation that the network will always work. Big stuff.

Knowing that I can do things I never imagined, I’m ready for Funky Monkey the Revolution

(OK, maybe not… that looks really hard)

Lesson #4: If you work as a team, you can accomplish your goal

Possibly the best part about open source is how people work together for a common goal — not just a group from a single company, but people from around the globe. In the ONAP project, it’s not unusual to be on calls with people from China, India, France, Canada, and the US all at once. Some of us are even competitors. But, we know that only by working together can we build great software and make automated NFV (Network Function Virtualization) a reality.

Knowing that not just my team, but all Tough Mudders will have my back, I’m ready to tackle the Pyramid Scheme.

So, wish me luck on Sunday. If my open source lessons fail me, then I guess it means I should’ve done more pullups!

This article originally appeared on LinkedIn.

 

All Things Open

Join The Linux Foundation at All Things Open; check out conference highlights below. (Image: All Things Open)

Going to All Things Open in Raleigh? While you’re there, be sure stop by The Linux Foundation training booth for fun giveaways and a chance to win one of two Raspberry Pi kits. Two winners will be chosen onsite on the last day of the conference, Oct. 24, at 3:05pm.

Other booth giveaways include The Linux Foundation branded webcam covers, The Linux Foundation projects’ stickers, Tux stickers, Linux.com stickers, as well as free ebooks: The SysAdmin’s Essential Guide to Linux Workstation Security, Practical GPL Compliance, A Guide to Understanding OPNFV & NFV, and the Open Source Guide Volume 1.

Be sure to check out these featured conference talks, including the Linux on the Mainframe session where John Mertic and Len Santalucia discuss how they’ve worked to create an open source, technical community where industry participants can collaborate around the use of the Linux and open source in a mainframe computing environment. And don’t miss ODPi’s session on the simplification and standardization of the Big Data ecosystem with common reference specifications and test suites.

Session Highlights

  • Accelerating Big Data Implementations For the Connected World – John Mertic
  • Advancing the Next-Generation Open Networking Stack – Phil Robb
  • Flatpak: The Portable, Secure Distribution of Desktop ApplicationsOwen Taylor
  • Intel: Core Linux Enabling Case Study and Demo
  • Integrating Linux Systems With Active Directory Using Open Source Tools – Dmitri Pal
  • Linux On the Mainframe: Linux Foundation and The Open Mainframe Project – John Mertic & Len Santalucia
  • Polyglot System Administration AKA: Don’t Fear the Other Language – Jakob Lorberblatt
  • The Next Evolution of The Javascript Ecosystem – Kris Borchers
  • The Revolution Will Not Be Distributed – Michael Hall
  • You Think You’re Not A Target? A Tale Of Three Developers – Chris Lamb

ODPi and Open Mainframe will also a have booth at All Things Open. Get your pass to All Things Open and stop by to learn more!

 

skydivers in the air

Don’t do fly-by contributions. Choose the right project, assign developers to consistently contribute upstream, and start to realize real influence and value from your open source investment.

When The Linux Foundation started OpenDaylight, our first networking project, nobody that I was working with had ever done open source before. Four years later there has been a significant shift in the entire networking industry. And we’re watching this transformation happen from one industry to the next.

In those four years, I’ve also witnessed many large organizations with significant engineering investments blunder their way into open source. For example, they might just fly in and drop 20,000 awesome lines of code into a project and then get upset that nobody actually picked it up.

The reality is that no matter how big and important your organization is, code doesn’t get picked up if you have no reputation in the open source community. The community must believe that the organization will actually be around to help support that and continue to evolve the code.

So if you want to be part of the industry transformation occurring now with open source — and your company’s future does depend on it — here are some basic steps for organizations new to open source to gain influence within an open source community.

Getting Value out of Open Source

First, let’s talk a little about why you want to gain influence in open source projects in the first place. There are three different tiers of value that individuals and organizations get from open source code.

  1. Take it and use it. (Great value.) It’s free, right? And there’s great value in that because you didn’t have to write the code. A lot of this software has been around for a long period of time, so you know it’s stable and it’s reliable, so it’s just a great resource.
  2. Customize it for your specific needs and contribute those changes back to the project. (Higher value.) You can continue to evolve with that project, pulling in changes that others have made that give great benefit to you as well. (See our guide to Participating in Open Source Communities for more.)
  3. Recognize this transformation occurring in your industry and rely on the platform that everybody in the industry is working on together. (Highest value.) When you start to lead feature sets and activities in those projects, you have a tremendous influence on what’s happening next.

The way we write these applications and the way we choose to implement them ultimately becomes the foundational aspects of our systems that we’ll be using for the next five, 10, 15, 20 years. To get the most value from open source within your organization, you want to help influence the direction of these foundational technologies.

Steps for Gaining Influence in Open Source

So the first step in building influence is: Pick the right project. Start by asking yourself, do I need to modify this code and what license is this code under? If it’s under a license that you can’t use in your particular situation, that code is not going to be applicable to you.

Next, if you need to modify the code (and often you don’t) and want the codebase to evolve with your changes, it’s very helpful to actually work with a community that would take your changes. You’ll need to look at who controls the project, what’s the governance, and who owns the copyright. Look at the project’s charter documents, join their IRC channel and their mailing list. Propose to do something and see what kind of a response comes back. Do the triage to figure out if it’s a project that’s going to work for you or not.

There are some communities that, even though it’s under an open source license, you have to assign your copyright over to some commercial entity, and only individuals from that organization are actually allowed to be committers. There are a variety of ways that projects can be structured that significantly inhibit your ability to make contributions. Understand if those hurdles exist and avoid projects that operate that way.

Everything within The Linux Foundation is structured in a way that allows for diverse contributions. We do that because we’re trying to get an industry to work together collaboratively to address an industry-wide challenge or opportunity.

But today, there is a certain amount of “open-source washing.” Some organizations will say “Hey, we’re going to put this into open source,” but it’s a significantly debilitated version of what they’re really trying to sell you in a commercial environment.

Now, at the same time, there are projects that are started by individual organizations where they really are trying to build a community and a collaborative environment. So just because the individual organization is sponsoring that activity doesn’t mean you shouldn’t engage, but you need to investigate.

Once you have chosen the right project, you will need to assign talented developers to work upstream. Very often when companies get into open source for the first time, they’ll create an entire group, 10, 15, 20 folks that are taking the code and doing stuff with it to augment it for their internal project and they’re not contributing anything back to the project. That invariably will get you into trouble six to 24 months down the road as the upstream open source project continues to evolve. The delta that you have created with your internal version becomes significantly disparate from the mainline code. Every time a new version comes out from the upstream, your team will spend more and more time actually back-porting their modifications into the upstream as opposed to doing new and meaningful development.

Put developers on the core of the project. So many organizations want to add new features that are going to be important to them, and that’s great. But if the core isn’t stable, reliable, and scalable then your features don’t really matter. Also, consider putting developers in support and infrastructure roles such as testing and release management or release engineering.  

Gaining Influence Through Credibility

All of this will help your organization gain credibility in the project community so that when your organization wants to do something down the road that might be a little off the wall, you end up having more influence and the ability to do that. It’s more likely that when you submit that 20,000 awesome lines of code the community will look at all this great work that your organization has done for this project and say, “Let’s support them in what they’re trying to do as best we can.”

Don’t be that organization that does fly-by contributions. Choose the right project, assign developers to consistently contribute upstream, and start to realize real influence and value from your open source investment.

 

The Tools for Managing Open Source Programs guide provides an exhaustive collection of categorized tools that any open source program can benefit from.

Is your organization looking to build out an open source program? If so, you’re not alone, but not every organization has a holistic sense of the available tools that can help create a healthy program. A simple charter document and a few spreadsheets for tracking projects won’t cut it anymore in managing a truly robust open source program. That’s where the new Tools for Managing Open Source Programs guide comes in. It can help any organization launch and maintain a thriving open source program.

“If you have more than 100 code repositories or 100 people that you’re trying to manage, you really can’t have someone doing it manually with spreadsheets anymore,” notes Jeff McAffer, Director of the Open Source Programs Office at Microsoft, in the guide. “Obviously, people still do it that way. But it starts to become ad-hoc and laborious. That’s where tools come into play. They allow you to scale.”

While launching and maintaining an open source program does require dedicated, task-specific tools, it is a mistake to assume that your organization must necessarily build its own tools from the ground up.

“Regarding existing tools and systems, my hope is that we’re quickly getting to a point where a company’s open source program office should not need to create any tools or technologies on their own,” said McAffer. “They should be able to find and use existing open source tools which can be used to manage their open source programs.”

Categorized Tools

The Tools for Managing Open Source Programs guide provides an exhaustive collection of categorized tools that any open source program can benefit from. These include Source Code Scanning and License Compliance tools, Bug Tracking tools, Release Management tools, and more. Are you familiar with FOSSology? It’s a Linux Foundation project that functions as an open source license compliance software toolkit capable of running license, copyright and export control scans from the command line. Have you heard of Docker Hub? It’s a cloud-based registry service that allows users to link to code repositories and build and test their images. These and many, many more useful tools are linked to and explained in the free guide.

Do you know how to answer questions like these?

  • How are your project APIs documented?
  • Have you laid out a Contributor Licensing Agreement that everyone can use?
  • Have you picked the right license for your project?

Various tools can help you determine the right answers to these questions, and the Tools for Managing Open Source Programs guide is a great way to surface them.

Methodologies

It’s important to understand that using open source for business strategy requires its own methodologies and processes which are very different than those needed when using and releasing proprietary software. As the guide notes:

“Nobody said it was going to be simple to move your company into the world of open source. But plenty of other companies, including giants like Microsoft and Google have done this before you and have provided detailed road maps, code, suggestions, and more to make your own journey easier. The creation of an open source program office and the selection of a package of critical tools to get your efforts started are within your grasp. By collaborating on open source projects and inviting others to collaborate with you, your company can gain immeasurable benefits and drive its progress forward with energy and innovation.”

The Tools for Managing Open Source Programs guide is one of a new collection of guides from The Linux Foundation and The TODO Group that are all extremely valuable for any organization setting up an open source program. The guides are available now to help you run an open source program office where open source is supported, shared, and leveraged. With such an office, organizations can establish and execute on their open source strategies efficiently, with clear terms.

These guides were not produced in a vacuum. Far from it. The advice you will find in them grew organically out of many interviews with some of the world’s leading open source experts. We encourage you to check out the guides and stay tuned for our continuing coverage of them.

Also, don’t miss the first article in this series, on How to Create an Open Source Program, which explores everything from the role of the open source program office to how successful open source programs at companies like Google function.

 

Concepts such as decentralizing strategy, delegating direction, and fierce transparency in communication are part of the backbone of successful open source projects. In my presentation at Open Source Summit EU in Prague, I will explore how these concepts are not only applicable to volunteer-run organizations but can also help growing corporations avoid some of the coordination overhead that often comes with growing teams and organizations.

We’ll look at some of the key aspects of how project members collaborate at The Apache Software Foundation (ASF). After that, we’ll take a closer look at German FinTech company Europace AG, which decided to move toward self-organization two years ago. We’ll highlight parallels between Europace AG’s organizing approaches and those of open source projects.

Let’s start with some of the core values of ASF projects.

Community over Code

One main principle is the concept of “community over code” — which means that without a diverse and healthy team of contributors to a project, there is no project. It puts the team front and center, as highlighted in the Apache project maturity model.

Meritocracy

Another core value to Apache projects is meritocracy — essentially meaning that there is no governance by fiat. There is no way to simply buy influence into projects — you have to invest time to gain influence. This directly translates to frequently given advice for how to get started with any given project: Focus on the things you are using yourself and fix what bothers you most essentially following a scratch your own itch kind of model. There is one level of indirection here: Committers and contributors can be paid either by their employers or as part of a consulting gig to increase their motivation to work on the topics that you are interested in.

Project independence

Be aware though that there is a strong principle of people participating as individuals on projects. That level of independence means that within an Apache project, there is no way to assign tasks to project members. Apache projects by design have a very flat organization with barely any hierarchy: Titles like Project Management Committee Chair (PMC Chair) are famous for coming with additional responsibility but no entitlement to task assignment. That means projects need a process for aligning people with potentially very diverse interests and itches to scratch. A key ingredient to coming up with workable decisions is making project goals and needs public and transparent — both in terms of asking for help and in showing appreciation and rewarding contributions that help the project. Another one can be found in an approach to integrating differing opinions and arguments in a “Yes, and” instead of a “Yes, but” fashion. This plays together very closely with the next concept.

Full transparency in open source projects

“What didn’t happen on the mailing list, didn’t happen” is one of the mantras for those involved with Apache projects. A high level of transparency is what makes Apache projects so easy to participate in across boundaries — whether they are geographic, corporate, or other. At Apache, mailing lists are treated as the point of reference for any decisions made. Thus, there is full transparency into the historic record of all open source projects.

Of course, with that entirely open and transparent model comes responsibility: A lot of decisions can be taken in the open. People tend to show better behavior when under public scrutiny. However, discussions involving interpersonal issues are best kept out of public sight. This is particularly important in a digital age where deleting statements made online and mirrored worldwide is pretty much impossible.

Project autonomy

Technologically, Apache projects are very diverse. However, when unified under one organization with a common focus on community, meritocracy, and communication patterns, there is a lot of freedom and decentralized decision making.

One important piece in growing the ASF as a whole was giving autonomy to each project. Still there are a couple topics that each project has to deal with: It’s great having one central entity answering questions on, for example. trademarks, licensing, or software patents. However, these entities controlling each of the 300 or so top level open source projects won’t scale. Instead, Foundation-level services serve as guidelines for Apache open source projects to manage their own activities in these areas. As a result, each project is supposed to have some people knowledgeable in those topics.

Open source patterns in the enterprise

In the past decade, people in the open source space have successfully brought several of these concepts to their respective employers — either in the form of open development, inner source, or open organizations. Meanwhile, ideas of transforming traditional hierarchical organizations into self-organized, open organizations found their way into concepts like sociocracy and holacracy.

Europace AG decided to move toward self-organization roughly two years ago. Looking at what was established during these two years, we see quite a few similarities to how nonprofits such as the Apache Software Foundation and The Linux Foundation work with open source projects:

Decentralization and autonomy

Europace AG is a tech company experienced with XP and Scrum, and in 2015, they began a journey toward self-organization, looking into holacracy and sociocracy frameworks. Organizationally, there are roughly 200 people split into four fairly autonomous technical units, each one of which is responsible for one business product of the company. Units are autonomous in how they self-organize: Some chose fairly standard Scrum-based software development in iterations and standard rituals. Others went for sociocratic or holacratic models. By organizing in circles, employees are given more influence over how decisions are made that affect their daily work. Backing for this level of autonomy can be found in the company’s own principles, where self-organization and decentralization are outlined as one of the four important values and are actively supported by the board.

This is not unlike Apache, where open source projects operate independently and autonomously in how they develop their software and self-organize (that is, provided they make sure that project governance remains independent of one single player, security issues are being dealt with, and legalities like licensing, patent issues, and trademarks are taken care of).

Transparent decision making

Everyone working for and with Europace AG is located in one timezone, and Europace AG itself has only one site. Nonetheless, there are good reasons to look at how decisions are made in distributed open source projects: Establishing a remote friendly communication model can help support employees with family, and deal with long commute distances. Apart from these obvious benefits, there are a few effects that aren’t quite obvious:

While an asynchronous communication model won’t lead to decisions happening faster, it can help with reducing the amount of interruptions caused by face-to-face meetings: By sharing a proposal before the actual meeting happens and providing a means to exchange ideas around that proposal can help with understanding people’s opinions, issues and suggestions with respect to the proposal. It can help with making that understanding transparent to everyone interested in the topic up front. Combined with a “Yes, and” philosophy, this can help form consent in the team quickly.

Organizational Groups

Splitting roughly 200 employees into just four units creates groups that are bigger than typical “2 pizza sized teams” often deemed ideal for software development teams. As a result, each unit had to figure out how to create sub-groups that can work well as teams.

A common approach was to create cross-functional feature teams, comprising both software developers and people familiar with financial domain (and people with backgrounds in data analytics, UX, etc. as needed). Those are called feature teams in anticipation of the fact that they aren’t necessarily particularly stable: Each week, there is a check point people can use to decide to change feature teams depending on where they are needed most. Decisions related to governance as well as product strategic aspects are delegated to the team itself. As a result, team members need a good understanding of the current product direction in terms of features and usage.

For tasks and topics that tend to fall under the general topic of “organizational operations” or “decision making with dependencies,” people organize themselves in circles, which are essentially groups of people interested in participating in making a decision related to a specific topic. Despite a lot of communication still happening face to face, some circles also set up a Slack channel that is open for all to join and participate. This level of transparency is crucial in elevating trust people have in those decisions.

Each of these circles can come with dedicated roles: Secretary being the one taking over meeting minute creation, a Lead Link making sure that “things will actually get done” and communicated to the people needing the respective information. While people can volunteer for these roles, at the end of the day these roles get filled through nomination by the team. This is not unlike Apache projects where new committers, PMC members, and even Foundation members are elected after nomination. On one hand, this kind of model shows a certain amount of appreciation for the work these people are doing. On the other, it helps with promoting people into these roles who according to their own self-assessment might not have confidence to take on the role.

Put the human first 

Much like Apache puts community development front and center with its open source projects, Europace AG puts team collaboration into a central position. This means that each employee is responsible for raising issues with team climate, and they are supported by a team of professional coaches, moderators, and mediators with a deep understanding of human communication.

Conclusion

When moving toward self-organization, corporations can learn a lot from how open source projects organize themselves. Europace AG is on the journey toward moving more power away from traditional formal structures, making it available for the people in purpose-driven, circle-based organization. It will be interesting to watch how Open Source project management, governance, and communication patterns can be applied within corporate contexts.

Isabel Drost-Fromm is Open Source Strategist at Europace AG Germany. She is a member of the Apache Software Foundation, co-founder of Apache Mahout and has mentored several incubating projects. Isabel is interested in all things FOSS, search and text mining with a decent machine learning background.