Posts

open source program

Gil Yehuda, Senior Director of Open Source at Oath (which owns the Yahoo and AOL brands), describes the company’s open source goals.

For seven years and counting, Gil Yehuda, Senior Director of Open Source at Oath Inc. (which owns the Yahoo and AOL brands), has led the open source program at Yahoo. Now with an expanded scope, he is gearing up to grow his team and improve the program. The company’s formal open source program office serves as a hub to connect all open source activities across the company, he says, but it didn’t start out that way.

As with many other companies, the open source program started informally with a group of diligent engineers and a few legal people. But the ad hoc group soon realized it needed a more formal program if it was going to be able to scale to address more issues and achieve specific business goals. With a formal program in place, they are poised to achieve its goals.

The top five of Oath’s numerous open source goals, according to Yehuda, are:

  1. Keep aligned with the industry on open source technology standards by avoiding creating unique tech stacks that Oath alone would have to manage at its own expense.
  2. Make it easy for engineers to interact with open source as users and as contributors.
  3. Be viewed as an open source friendly company for partnerships and collaborations.
  4. Be known as a great place for engineers to work on open source projects.
  5. Give back to the Open Source community by sharing code and practices.

Measuring and monitoring success requires the right tools and attitudes. Yehuda says at Oath they actively solicit and listen to the needs of their many engineering teams, track all their work transparently in Jira, and spread the work across many teams who help with the process.

“We have custom tools we use to check code and manage projects, but we’re hoping to work more with our peers in the TODO Group on tooling we can share across many of our peer open source program offices,” he said.

Success comes from being open, at scale

Yahoo helped make Apache Hadoop the cornerstone of the big data revolution when it took the early code and created a team around it to help it scale to Internet-scale. They agreed to publish it all as open source. When the need for real-time processing came to the forefront, Yahoo created S4 and open sourced it too, but then discovered Storm was just published, too, and it looked more promising. The team ditched their own code and put their efforts into helping make Storm even better.  

“We applied to Apache Storm what we learned from Hadoop and S4,” Yehuda said. “Our goal was to make it great, even though it kind of competed with our own first stab.”

Storm is a success today, and the company runs it alongside Hadoop to power many of its products. They added machine learning and high-scale data serving capabilities by adding Vespa Engine, to their platforms, and then published that too. And they helped other machine learning projects scale better too, all by publishing open source.

“We’ve leveraged our expertise with Storm to help both Caffe and TensorFlow achieve better scalability. We don’t own these solutions exclusively. Rather we share our code and help others — all the while we get to leverage our expertise to build one of the industry’s most scalable platforms for our use,” he said. “This saves us money while making us a fantastic place to work on projects that impact hundreds of millions of people.”

The program office worked on strategy, legalities, and execution of these and similar projects. Leveraging open source licensing and processes effectively was a key element throughout. Now as Oath, this work continues and expands.

Yehuda cited three key lessons he learned managing an open source program:

  1. Be a service to the engineers, not a barrier.
  2. Accept that challenges will be never-ending.
  3. Run the program office like you run an open source project: Be transparent in the way decisions are made and be open to input and collaboration from everyone.

“There are so many edge cases that come up — partnerships, acquisitions, unclear contract terms — and we simply need to be open to learn, explore, and come up with an answer to every open source related question. But the most rewarding part of my job is when people tell me they joined our company because they knew about our open source friendly culture. You know, we’re always looking for open source talent, and I’m hiring into the program office.” added Yehuda.

open source program

“We believe that our projects help move the industry forward while giving other companies and individuals the opportunity to use our platform to scale more quickly and build better products.” – Christine Abernathy, Open Source Developer Advocate at Facebook.

Facebook’s open source team was “formally” created in 2009, but the company has built with open source from its inception. Facebook.com was originally built on top of the LAMP (Linux/ Apache/ MySQL/ PHP) stack. And over time Facebook has used and contributed back to these projects, as well as evolved and released new projects such as Hack which has its roots in PHP.

“Open source is core to our engineering DNA. We believe that sharing our code and even hardware designs accelerates the pace of innovation in the world. We believe that our projects help move the industry forward while giving other companies and individuals the opportunity to use our platform to scale more quickly and build better products.” – Christine Abernathy, Open Source Developer Advocate at Facebook.

Custom tools to manage open source

Facebook has a dedicated Tools team within the open source program office that is responsible for building internal tools to help manage its open source portfolio. This includes the projects that Facebook shares, which are mostly hosted on GitHub, as well as the other external projects they contribute to such as the Linux Kernel.

The program office provides a dashboard for each project that includes GitHub metrics such as the number of open issues or the ratio of internal to external contributions for a given time period. Project maintainers are given tools so they can bring GitHub pull requests and issues into their internal review and bug tracking systems. This makes it easier for engineers to manage external issues where they’re most comfortable. Maintainers also have access to workflow tools to reliably push internal commits out to GitHub, making it easy to quickly sync internal and external code bases and reducing the churn on landing external contributions.

The open source team can look at these project dashboards to help analyze the health of a particular project. They’ve even open-sourced some of these workflow tools: mention-bot and FBShipIt.

The team collects top-level statistics on how the overall portfolio is doing through aggregate dashboards across the GitHub orgs they manage. These are used to provide high-level reports to stakeholders and a community of internal open source enthusiasts. The tools team also provides insight into top contributors. Project maintainers are encouraged to refer to this list and reward their top contributors. The company periodically thanks its top internal contributors and makes some of this information available to its internal review systems.

The open source office also provides tooling to guide potential projects through the review process. This helps streamline the process and helps the team easily spot and correct bottlenecks.

The open source team also provides services such as documentation. This includes helping out with the technical content as well as building out some of the documentation infrastructure and templates that projects can use.

Open Source Success Through Steady Progress

At the end of each half the program office identifies goals around metrics they want to achieve. The metrics they track include:

  • The average age of open issues or open pull requests
  • The ratio of external to internal commits
  • The number of commits
  • The growth in followers and forks
  • The number of social media followers.

They’re periodically tweaking what they measure as they refine what it means to maintain a healthy portfolio.

Facebook also surveys new hires every six months to gauge their awareness of its open source program. They set baseline metrics a few surveys back and the goal is to maintain or grow those numbers.

Their open source success isn’t the result of one action but the cumulative effect of a steady stream of quality releases over the years and a focus on growing thriving communities to support those projects.

“Projects like React, React Native, Create-React-App, Immutable, HHVM, Fresco, and GraphQL are the constant beat that have contributed to the success of our program,” Abernathy said.

One of Facebook’s most successful projects is React Native. It makes use of many of Facebook’s tools to help manage the community. For example, mention-bot came out of this project and was a way to quickly identify reviewers for a pull request. FBShipIt helped it cut down the time to bring in external contributions, review them internally, and land these contributions back out to GitHub. In the early days, this process sometimes took a day as much of it is manual. Now this can be done in as little as minutes if it’s an automatic reviews.

The open source program office also provided documentation services to help refresh and keep the React Native site up to date.

Tips for New Open Source Program Managers

Organizations that are just establishing an open source strategy and program office can learn from the success of Facebook’s open source program. Here are the three key practices that Abernathy shared that have contributed to their success as a program office:

  1. When evaluating what to share, it should be something that’s useful to your company. Many of the projects that Facebook shares are used in production and include all the benefits that come along with that. This means those projects are likely to have continued support which in turn means the community is well supported.
  2. Find a way to highlight, promote, and reward your open source contributors, both internal and external. Facebook has periodic reports that highlight its open source heroes. This helps raise the profile of engineers and their work with managers who may sometimes not be managers of that open source project.
  3. As a central program office, find pain points that cut across the various projects and tackle them.

For example, many projects had previously built their own commit copying scripts and it was the number one pain point from a survey they ran at that time. FBShipIt, which copies commits between repositories, was built to address this and it’s owned by the open source team. It moved the burden off the engineering teams and is universally praised for helping smooth the workflow for pulling in external contributions.

Acknowledgments

For this feature we interviewed Christine Abernathy (@abernathyca), Open Source Developer Advocate at Facebook, to learn more about the Facebook’s open source program. Libby Clark performed the interview.

At organizations of all types, launching and maintaining successful open source programs has become a business priority. A strong open source program office helps to ensure that open source is supported, nurtured, shared, explained, and leveraged. With such an office, organizations can establish and execute on their open source strategies in clear terms.

With all this in mind, The Linux Foundation and The TODO Group (Talk Openly Develop Openly) have published a free collection of detailed open source guides to aid companies developing open source programs. The guides are available to you now, and this is the first in a series of articles that can introduce you to the value of the guides.

How to Create an Open Source Program is the first of the guides, and it explores everything from the role of the open source program office to how successful open source programs at companies like Google function. The guide also includes insights and advice from open source experts, including John Mark Walker, Founder of the Open Source Entrepreneur Network, and Will Norris, Open Source Office Manager at Google.

“The open source program office is an essential part of any modern company with a reasonably ambitious plan to influence various sectors of software ecosystems,” notes Walker, in the guide. “If a company wants to increase its influence, clarify its open source messaging, maximize the clout of its projects, or increase the efficiency of its product development, a multifaceted approach to open source programs is essential.”

The How to Create an Open Source Program guide makes clear that there is not a one-size-fits-all approach to creating a successful program. In fact, Google’s Norris notes that stakeholders from individual business units play a key role in how open source projects advance at Google.

“We allow the various business units around the company to make the decision on whether it makes sense to open source a given project from a business perspective, because there’s a lot of different reasons why you might open source a project or a piece of code,” he notes. “We’re comfortable with allowing projects to take the approach that works for them given their goals. We play more of a role of facilitating and advising.”

The first guide lays out recommendations for how to include stakeholders ranging from Legal to Engineering in the maintenance of a program office. It also delves into the importance of setting clear program policies and observing compliance guidelines.

“Having a well-defined policy in place, that’s great, but it’s got to be a well-defined minimal policy,” said Jeff Mcaffer, director of the Open Source Programs Office at Microsoft, who was interviewed for the first guide. “Otherwise you get lawyers, security folks, business folks, all piling in their concerns and constraints. Soon you end up with a straitjacket full of policy that basically means that nobody can do anything.”

These free guides are extremely valuable for any organization setting up an open source program. Notably, the guides were not produced in a vacuum. Far from it. The advice you will find in them grew organically out of many interviews with some of the world’s leading open source experts. We strongly encourage you to check out the guides, and stay tuned to this space for more articles in this series.