Posts

In its role as an ISO PAS submitter, JDF and LF now can move from idea to code, to standard, to an internationally recognized standard, vastly improving the reach and availability of the technologies created by our amazing communities.

Introduction

This week, we are proud to announce that the Joint Development Foundation (JDF), which became part of the Linux Foundation family in 2019, has been accepted as an ISO/IEC JTC 1 PAS (“Publicly Available Specification”) Submitter. The OpenChain Specification is the first specification submitted for JTC 1 review and recognition as an international standard. 

The JDF was formed to simplify the process of creating new technical specification collaboration efforts.  Standards and specifications are vitally important for the creation or advancement of new technologies, ensuring that the resulting products are well defined, provide predictable performance and that different implementations can interoperate with one another.  

Why the Linux Foundation cares about standards

The Linux Foundation itself was formed out of the merger of the Free Standards Group, which maintained the LSB (“Linux Standards Base”) and the Open Source Development Labs. Open standards and open source software have been part of the mission from the very beginning.

Standards play a role in everyone’s life. Think about the things you touch every day, as simple as a power plug, the USB connector on your phone or laptop, or the WiFi that you use in your business and your home to connect your mobile devices wirelessly. All of these devices need to be able to interoperate with each other.

A pragmatic and sensible approach to solving interoperability issues would be to create open source software projects everyone can use. However, there are cases where open source software alone will not solve all the implementation challenges that open standards can achieve. 

Open source software in and of itself may not solve particular situations where there will be many implementations in many different device or delivery models (e.g., video codecs or 3D printer designs with many software design tools and many hardware printers and scanners). Still, in other cases, that fragmentation is due to different device capabilities, implementation details, or limitations that open source software cannot resolve alone.

The design and capacities of many things are defined by industry stakeholders as a standard so that every plug and device is interoperable and capable of the same connectivity.  Every country in the world has its own national standards bodies that define the standards it deems necessary, from power transmission, radio spectrum, food safety, and others.

Not all standards bodies are national standards bodies, with standards organizations coming in many shapes and sizes. Many standards are developed by industry-specific organizations that have a common set of technical objectives and are seeking a common set of use cases, a shared set of key design and performance criteria, and a common test specification to ensure interoperability.  

For the Linux Foundation, our collaborations can range in size from small to large, but their impact can extend internationally. There is not a Linux kernel per country or an Open Container Initiative specification per country, and so on. The world is dependent on our communities.

Like Linux Foundation source code projects, JDF standards and specification development projects can range from small, industry-specific efforts, to large multi-industry collaborations. And it is the JDF’’s goal to serve these various communities.  By obtaining PAS status, JDF can help specification and standards communities ranging from the smallest collaborations through to international standardization.   

How Open Standards differ from Open Source projects

Open standards are best defined as specifications made available to the public, which are developed and maintained via an inclusive, collaborative, transparent, and consensus-driven process. Open standards facilitate interoperability and data exchange among different products or services and are intended for widespread adoption.

Open source software is defined by the OSI’s Open Source Definition. In practice, we generally care more about communities that form to work on open source software in a public, transparent collaboration where the code evolves over time to address new use cases, features, requirements, and gaps.  

Sustainable open source software communities also see continuous improvements as bugs and security issues are identified and fixed. Open source code is typically created as a collaborative effort in which programmers improve upon the code and often share the changes among the programming community for such projects. At a high level, open source licenses allow users the freedom to use, modify, and distribute the source code without requiring any further permission.

So, for example, software such as the Linux kernel is open source software in an open community, whereas the IETF curates open standards that enable the world to connect through an open Internet.

Another excellent example of how standards come into play across different hardware and software platforms are web servers. There are many web server platforms, both open source, and proprietary — such as Apache’s and Microsoft’s IIS. Some are optimized for speed, others for large deployments, some for low power devices, and for other applications. But as long as they can all speak HTTP (and other standards), they can still all communicate across the spectrum of devices.

The process of creating standards

Standards bodies are usually formed by industry stakeholders to support the activities needed to develop a specific solution to a common problem. The resulting solution is generally referred to as a specification, a blueprint for building an implementation of a solution to the problem. In some cases, the same group may also create an open source implementation, but the implementation will be specific to a set of use cases and requirements.

A standards body is the legal organization often created to provide a neutral home to the collaboration, including financial and legal support, guardrails against antitrust issues, managing copyrights and other intellectual property terms that might bear on the specification. Many will say the most important role of a standards body is to provide a neutral governance model that enables inclusive participation from all parties, where no one organization controls the specification.

The challenges in creating specifications

For something as crucial as a specification, the process of creating a specification setting body can be complicated.  

And even when the participants are aligned, the devil is always in the details. The negotiations to establish a new standards organization often involves hundreds of hours of lawyer time and a method of negotiating the nuances of the working rules and the license terms for copyrights, patents, and trademarks related to the effort.  The entire process can take many months — and it’s a requisite precursor in most cases to the technical contributors getting started. So before anyone knows what the output will be, or if it will even work, many organizations collectively invest thousands to millions of dollars on months of negotiations that delay the start.

Once the mass negotiation is done, the legal entity needs to file for non-profit status, set up bank accounts, set up accounting, finance, and HR operations, collect fees from its members, and file its taxes, just like a commercial company. These activities need to occur even if all the initiating organizations are 100% aligned on the need for the specification. Once that is all done, the engineers can get together to develop a specification, often a year after the initial idea was created.

The JDF was founded to make the entire process of forming a new standards body faster, and remove the negotiations. The JDF has created a set of default terms that reflect industry best practices and proven widely accepted legal terms.  By providing a choice of pre-existing, industry-accepted terms, JDF replaces custom negotiation with a “check the box” model. This model adopts best practices while giving flexibility through a few commonly known choices to the founders about essential terms such as copyright, intellectual property licensing, source code licensing, and governance structures.  It also allows JDF projects to be customized to meet the needs of the community, without resorting to time-consuming line-by-line negotiations.  

And once those terms are in place, the new project is formed as an entity under the non-profit JDF.  In combination with world-class operational support programs, a new project can get started in a matter of days, with resources ready to go, rather than the months to the years-long process required to form a traditional standards body. The cost of this effort is so low that a specification project can be established without any funding needed for the creation or ongoing entity management.

In essence, the JDF provides a “standards organization in a box.” Just pick a few menu options, give the effort a name and off you go creating specifications. 

The net impact of the JDF process means that companies with the need to collaborate can form the project, define the technical scope and begin inviting engineers to contribute to the project in a matter of days with minimal friction.

Internationally recognized standards through the ISO/IEC JTC 1 PAS process

One method of recognizing international standards is via the ISO/IEC JTC 1 PAS (Publicly Available Specification) Process. Once accepted through this process, the specification is recognized as an international standard. 

ISO is an independent, non-governmental international organization with a membership of 164 national standards bodies, and its standards are among the most universally recognized and accepted throughout the world.  

The IEC (International Electrotechnical Commission) is the world’s leading organization for the preparation and publication of International Standards for all electrical, electronic, and related technologies. 

ISO and IEC joined together to create ISO/IEC JTC 1, which is the international group dedicated to developing worldwide Information and Technology (ICT) standards. JTC 1 has been responsible for many key IT standards — including video compression technology and programming languages, among many others.

The Publicly Available Specification (“PAS”) process was created by a collaboration between ISO/IEC JTC 1 to allow for transposition of technical specifications from recognized standards bodies, which will enable them to become an ISO/IEC recognized standard. 

PAS Submitters must first be approved after a review of an extensive set of criteria by the external standards bodies. Once approved, a PAS Submitter may put forward some of its specifications (the publicly available specifications, PAS) to JTC 1 for national body approval and thereby international recognition. 

And once ISO/IEC JTC 1 approves a PAS submission, it becomes an international standard.

The JDF’s acceptance as a PAS Submitter is vital to the industry because it reduces friction on the path from great ideas, to well-formed technical specifications, to international recognition of the best of those specifications. JDF has the responsibility for ensuring that the process of creating the specifications is rigorous, inclusive, and conforms to the quality standards set by ISO/IEC JTC 1. The benefit of having a professionally managed standards organization like JDF is that we help ensure those requirements are met.  

And it also means that JDF provides a capability that few other organizations can — a path for communities to start from a small collaboration and grow to become an international standard.  

Understanding the OpenChain specification, our first PAS submission

The OpenChain Specification identifies the key requirements of a quality open source compliance program. It is intended to foster a software supply chain where open source is delivered with trusted and consistent compliance information. It provides a clear way to achieve effective management of open source for software supply chain participants, such that the requirements and associated collateral are developed collaboratively and openly by representatives from the software supply chain, open source community, and academia.

“The OpenChain Project is a clear example of cooperative development to share a common challenge,” says Shane Coughlan, OpenChain General Manager. “Hundreds of companies have come together, shared knowledge, and built a clear, focused industry standard based on their experience. The result is a compact but effective standard suitable for companies of all sizes in all markets.”

The OpenChain Specification has been in the market since late 2016 and has seen increasingly broad adoption to-date. The OpenChain participants include national user groups exceeding 100 participants and over 3,500 subscribers to the primary communication channel mailing list. ISO/IEC JTC 1 recognition will help to guide the evolution of the specification from de facto to de jure standard, and in the process assist procurement, sales, and other departments around the world adopt and manage OpenChain specification-related activities easily.

Conclusion

With its recognition as a PAS Submitter, JDF now provides the broadest range of support to standards communities – from small collaborations to those seeking international standards. As part of the Linux Foundation family, JDF is providing communities with new ways to collaborate.  

By affiliating with JDF, the Linux Foundation ecosystem can benefit from the support and expertise to move open source specifications into an open standards-track, that empowers engineers and developers to collaborate in the creation of a specification and standard. By using this new submissions process, they can take their standard a step further to achieve international recognition. Conversely, the importance of the JDF joining the Linux Foundation family is significant because it is in alignment with the organization’s overall goal of furthering the commitment to neutral governance and alignment of open source software and open standards.

— Jim Zemlin, Executive Director, The Linux Foundation

SPDX License Identifiers can be used to indicate relevant license information at any level, from package to the source code file level.

Accurately identifying the license for open source software is important for license compliance. However, determining the license can sometimes be difficult due to a lack of information or ambiguous information. Even when there is some licensing information present, a lack of consistent ways of expressing the license can make automating the task of license detection very difficult, thus requiring significant amounts of manual human effort.   There are some commercial tools applying machine learning to this problem to reduce the false positives, and train the license scanners, but a better solution is to fix the problem at the upstream source.

In 2013,  the U-boot project decided to use the SPDX license identifiers in each source file instead of the GPL v2.0 or later header boilerplate that had been used up to that point.   The initial commit message had an eloquent explanation of reasons behind this transition.

Licenses: introduce SPDX Unique Lincense Identifiers


Like many other projects, U-Boot has a tradition of including big

blocks of License headers in all files.  This not only blows up the

source code with mostly redundant information, but also makes it very

difficult to generate License Clearing Reports.  An additional problem

is that even the same lincenses are referred to by a number of

slightly varying text blocks (full, abbreviated, different

indentation, line wrapping and/or white space, with obsolete address

information, ...) which makes automatic processing a nightmare.


To make this easier, such license headers in the source files will be

replaced with a single line reference to Unique Lincense Identifiers

as defined by the Linux Foundation's SPDX project [1].  For example,

in a source file the full "GPL v2.0 or later" header text will be

replaced by a single line:


        SPDX-License-Identifier:        GPL-2.0+


We use the SPDX Unique Lincense Identifiers here; these are available

at [2].

. . .


[1] http://spdx.org/

[2] http://spdx.org/licenses/

The SPDX project liked the simplicity of this approach and formally adopted U-Boot’s syntax for embedding SPDX-License-Identifier’s into the project.  Initially, the syntax was available on the project WIKI and was formalized in SPDX specification version 2.1 “Appendix V: Using SPDX short identifiers in Source Files”.  Since then,  other upstream open source projects and repositories have adopted use of these short identifiers to identify the licenses in use, including github in its licenses-API.  In 2017, the Free Software Foundation Europe created a project called REUSE.software  that provided guidance for open source projects on how to apply the SPDX-License-Identifiers into projects.   The REUSE.software guidelines were followed for adding SPDX-License-Identifiers into the Linux kernel, later that year.

The SPDX-License-Identifier syntax used with short identifiers from the SPDX License List short form identifiers (referred here as SPDX LIDs) can be used to indicate relevant license information at any level,  from package to the source code file level. The “SPDX-License-Identifier” phrase and a license expresssion formed of SPDX LIDs in a comment form a precise, concise and language neutral way to document the licensing, that is simple to machine process.  This leads to source code that is easier to read, which appeals to developers, as well as enabling the licensing information to travel with the source code.

To use SPDX LIDs in your project’s source code,  just add a single line in the following format, tailored to your license(s) and the comment style for that file’s language.  For example:

// SPDX-License-Identifier: MIT

/* SPDX-License-Identifier: MIT OR Apache-2.0 */

# SPDX-License-Identifer: GPL-2.0-or-later

To learn more about how to use SPDXLIDs with your source code,  please see the guidance in the documentation in the SPDX project, REUSE.software  or David Wheeler’s SPDX tutorial.    

In addition to U-boot and Linux transitioning to use the SPDXLIDs,  newer projects like Zephyr and Hyperleger fabric have adopted them right from the start as a best practice.   Indeed, to achieve the Core Infrastructure Initiative’s gold badge, each file in the source code must have a license, and the recommended way is to use an SPDX LID.  

The project MUST include a license statement in each source file. This MAY be done by 
including the following inside a comment near the beginning of each file: 
SPDX-License-Identifier: [SPDX license expression for project].

When SPDX LIDs are used,  gathering license information across your project files can start to become as easy as running grep. If a source file gets reused in a different package,  the license information travels with the source, reducing the risk of licence identification errors, and making license compliance in the recipient project easier.  By using SPDX LIDs in license expressions, the meaning of license combinations is understood more accurately. Saying “this file is MPL/MIT” is ambiguous, and leaves recipients unclear about their compliance requirements. Saying “MPL-2.0 AND MIT” or “MPL-2.0 OR MIT” specifies precisely whether the licensee must comply with both licenses, or either license, when redistributing the file.

As illustrated by the transition underway in the Linux kernel,  SPDX LIDs can be adopted gradually. You can start by adding SPDX LIDs to new files without changing anything already present in your codebase.  A list of projects known to be using SPDX License Identifiers can be found at: https://spdx.org/ids-where,  and if you know of one that’s missing,  please send email to outreach@lists.spdx.org.  

Learn more in this presentation at Open Source Summit: Automating the Creation of Open Source BOMs

Open Data

NOAA is working to make all of its data available to an even wider group of people and make it more easily understood (Image: NOAA).

The goal of the National Oceanic and Atmospheric Administration (NOAA) is to put all of its data — data about weather, climate, ocean coasts, fisheries, and ecosystems – into the hands of the people who need it most. The trick is translating the hard data and making it useful to people who aren’t necessarily subject matter experts, said Edward Kearns, the NOAA’s first ever data officer, speaking at the recent Open Source Leadership Summit (OSLS).  

NOAA’s mission is similar to NASA’s in that it is science based, but “our mission is operations; to get the quality information to the American people that they need to run their businesses, to protect their lives and property, to manage their water resources, to manage their ocean resources,” said Kearns, during his talk titled “Realizing the Full Potential of NOAA’s Open Data.”

He said that NOAA was doing Big Data long before the term was coined and that the agency has way too much of it – to the tune of 30 petabytes in its archives with another 200 petabytes of data in a working data store. Not surprisingly, NOAA officials have a hard time moving it around and managing it, Kearns said.

Data Sharing

NOAA is a big consumer of open source and sharing everything openly is part of the organization’s modus operandi. On a global level, “the agency has been a leader for the entire United States in trying to broker data sharing among countries,” Kearns said. One of the most successful examples has been through the United Nations, with an organization called World Meteorological Organization (WMO).

Agency officials have a tendency to default making their products accessible in the public domain, something Kearns said he’d like to change. By adopting some modern licensing practices, he believes the NOAA could actually share even more information with the public. “The Linux Foundation has made progress on the community data license agreement. This is one the things I’d like to possibly consider adopting for our organization,’’ he added.

One of the great success stories the NOAA has in terms of getting critical data to the public was after Hurricane Irma hit Florida in September 2017, he said.

“As you can imagine, there were a lot of American citizens that were hungry for information and were hitting the NOAA websites very hard and data sites very hard,’’ he said. “Typically, we have a hard time keeping up with that kind of demand.” The National Hurricane Center is part of the NOAA, and the agency took the NHC’s website and put it on Amazon Cloud.

This gave the agency the ability to handle over a billion hits a day during the peak hurricane season. But, he continued, “we are still … just starting to get into how to adopt some of these more modern technologies to do our job better.”

Equal Access

Now the NOAA is looking to find a way to make the data available to an even wider group of people and make it more easily understood. Those are their two biggest challenges: how to disseminate data and how to help people understand it, Kearns said.

“We’re getting hammered every day by a lot of companies that want the data… and we have to make sure everybody’s got an equal chance of getting the data,” he said.

This is becoming a harder job because demand is growing exponentially, he said. “Our costs are going up because we need more servers, we need more networks,” and it’s a problem due to budget constraints.

The agency decided that partnering with industry would help facilitate the delivery of data.

The NOAA is going into the fourth year of a deal it signed with Amazon, Microsoft, IBM, Google, and a nonprofit out of the University of the Chicago called the Open Commons Consortium (OCC), Kearns said. The agreement is that NOAA data will remain free and open and the OCC will host it at no cost to taxpayers and monetize services around the data.

The agency is using an academic partner acting as a data broker to help it “flip this data and figure out how to drop it into all of our collaborators’ cloud platforms, and they turn it around and serve many consumers from that,” Kearns explained. “We went from a one-to-many, to a one-to-a-few, to a many model of distribution.”

People trust NOAA’s data today because they get it from a NOAA data service, he said. Now the agency is asking them to trust the NOAA data that exists outside the federal system on a partner system.

On AWS alone the NOAA has seen an improvement of over two times the number of people who are using the data, he said. The agency in turn, has seen a 50 percent reduction in hits on the NOAA servers.

Google has loaded a lot of the agency’s climate data to its BigQuery data warehouse, “and they’ve been able to move petabytes of this data just in a few months, just because the data now has been loaded into a tool people are already using.”

This “reduces that obstacle of understanding,’’ Kearns noted. “You don’t have to understand a scientific data format, you can go right into BigQuery… and do analyses.”

Data Trust

Being able to trust data is also an important component of any shared initiative, and through the NOAA’s Big Data Project, the agency is seeking ways of ensuring that the trust that comes with the NOAA brand is conveyed with the data, he said, so people continue to trust it as they use it.  

“We have a very proud history of this open data leadership, we’re continuing on that path, and we’re trying to see how we can amplify that,’’ Kearns said.

NOAA officials are now wondering if the data is being made available through these modern cloud platforms will make it easier for users to create information products for themselves and their customers.

“Of course, we’re also looking for other ways of just doing our business better,’’ he added. But they want to figure out if it makes sense to continue this experiment with its partners. That, he said, they will likely know by early next year.

Watch the complete presentation below:

Fossology

To help celebrate Fossology’s 10th anniversary, we look at how the project makes it easier to understand and comply with open source licenses.

FOSSology turns ten this year. Far from winding down, the open source license compliance project is still going strong. The interest in the project among its thriving community has not dampened in the least, and regular contributions and cross-project contributors are steering it toward productive and meaningful iterations.

An example is the recent 3.2 release, offering significant improvements over previous versions, such as the import of SDPX files and word processor document output summarizing analysis information. Even so, the overall project goal remains the same: to make it easier to understand and comply with the licenses used in open source software.

There are thousands of licenses used in Open Source software these days, with some differing by only a few words and others pertaining to entirely different use universes. Together, they present a bewildering quagmire of requirements that must be adhered to, but only as set out in the appropriate license(s), the misunderstanding or absence of which can revert rights to a reserved status and bring about a complete halt to distribution.  

How FOSSology came to be

In short, there are over a 1000 different ways licensing can go mildly or horribly wrong, creating a desperate need to find one single way to make sure everything goes consistently right. Enter FOSSology, which as the website points out, is all about scanning: It’s a framework, toolbox and Web server application for examining software packages in a multi-user environment.

There are several important highlights since the first version of the FOSSology project was published in December 2007. The Linux Foundation started hosting it in 2015. The 3.2 release was in March 2018, which, as mentioned above, provides the ability to import SPDX files. SPDX (Software Package Data Exchange) is another Linux Foundation project that helps reduce complexity by defining standards for reporting and sharing licensing information. FOSSology is the first open source project to consume SPDX in this way.

“This project has been more successful than anticipated, because license compliance was a very special topic, and running it as an open source project is also difficult, because it has a naturally small community,” said Michael C. Jaeger, Senior Research Scientist Open Source Software at Siemens AG, Maintaining FOSSology and SW360, and Trainer at SW Compliance Academy.

When goal and delivery are tightly entwined, as are the benefactors and beneficiaries, good things come from any project.

“License compliance for open source projects is hard, and FOSSology helps here by doing most of the work, such as scanning the files to find licenses, copyright statements and more, to simplify the necessary clearing. It also generates reports which can be used to document the results, which is rather important in the context of larger companies,” said Maximilian Huber, a software consultant at TNG Technology Consulting.

A paper titled “The FOSSology Project: 10 Years Of License Scanning,” has been prepared to commemorate the 10th anniversary. Project members will be participating  at the FSFE’s Legal and Licensing Workshop in Barcelona this week to present on the project.

The project’s value and who benefits  

“It is important because it offers organizations a free software solution for license compliance – an area where commercial products have a very dominant position for more than a decade. However, with free software, especially open source projects can implement license compliance without upfront cost,” explained Jaeger.

FOSSology fits in well with the other open source compliance related projects like SPDX, OpenChain, and SW360. Indeed, there is even community and developer cross-over with some of these projects and FOSSology.

“There is one person who is a maintainer in both the SW360 and FOSSology projects, and there are some persons contributing to both projects in different roles,” said Jaeger. “Consequently and naturally, there is good coordination between both projects. The FOSSology project also has a long history for supporting SPDX since it represents the de facto standard for exchanging license compliance information.”

“With its review functionality, FOSSology was one of the first supporters of the concept of concluding a license in SPDX,” he added. “It was also the first project which allowed for importing SPDX descriptions, another elementary support because the “X” in SPDX stands for exchange and not “eXport.” As far as I know, OpenChain is not concerned very much with particular tooling; however, FOSSology helps to implement OpenChain conformance.”

And the momentum continues. More changes are on the horizon and some new obstacles as well.

“In the future, more and more open source projects will be straightforwardly licensed and the strong scan correction functionality and file review functionality of FOSSology will move to the background,” said Jaeger.

“However, questions still arise because of incompatibilities of licensings, or in considering obligations of licensing. Therefore, FOSSology needs to shift its focus from correcting scan results of not-well-formed licensing to licensing analysis and license problems on the component level.”

Modernization efforts are also under consideration.

“An important goal is to modularize the parts of FOSSology, to allow a smooth transition to a more modern architecture and software stack,” added Huber.

As even more licensing and related tools cross the horizon, simplifying the information exchange between them and FOSSology will be an ongoing task. That in turn will further cement FOSSlogy’s place in the license compliance ecosystem.

But today, all attention is on a decade of successes and the community that’s responsible for so many wins.

Happy anniversary, FOSSology!

Modern open source projects rarely consist solely of all new code, written entirely from scratch. More often, they are built from many sources. And, each of these original sources may operate under a particular license – which may also differ from the license that the new project uses.

license scanning and complianceA new publication, called License Scanning and Compliance Programs for FOSS Projects, aims to clarify and simplify this process. This paper, written by Steve Winslow from The Linux Foundation, describes the benefits of license scanning and compliance for open source projects, together with recommendations for how to incorporate scanning and compliance into a new or existing project.

Winslow runs The Linux Foundation’s license scanning and analysis service, and he advises projects about licenses identified in their source code and dependencies.

He says that getting license compliance right early can help attract contributors and users to an open source project. However, he notes that license scanning and compliance are not end goals; rather, they are processes that can serve other objectives, including:

  • Protecting the project’s developers.
  • Assisting downstream compliance efforts.
  • Demonstrating project maturity.  

According to Winslow, “any project that implements license scanning and compliance should aim to make it sustainable” and should set realistic goals to avoid being overwhelmed by the number of options and issues that may arise.

Winslow also explains how using tools, such as FOSSology for license scanning and Software Package Data Exchange (SPDX) to help package scan results into meaningful reports, can help projects succeed in compliance efforts.

Learn more and download this free publication now.

At organizations of all types, launching and maintaining successful open source programs has become a business priority. A strong open source program office helps to ensure that open source is supported, nurtured, shared, explained, and leveraged. With such an office, organizations can establish and execute on their open source strategies in clear terms.

With all this in mind, The Linux Foundation and The TODO Group (Talk Openly Develop Openly) have published a free collection of detailed open source guides to aid companies developing open source programs. The guides are available to you now, and this is the first in a series of articles that can introduce you to the value of the guides.

How to Create an Open Source Program is the first of the guides, and it explores everything from the role of the open source program office to how successful open source programs at companies like Google function. The guide also includes insights and advice from open source experts, including John Mark Walker, Founder of the Open Source Entrepreneur Network, and Will Norris, Open Source Office Manager at Google.

“The open source program office is an essential part of any modern company with a reasonably ambitious plan to influence various sectors of software ecosystems,” notes Walker, in the guide. “If a company wants to increase its influence, clarify its open source messaging, maximize the clout of its projects, or increase the efficiency of its product development, a multifaceted approach to open source programs is essential.”

The How to Create an Open Source Program guide makes clear that there is not a one-size-fits-all approach to creating a successful program. In fact, Google’s Norris notes that stakeholders from individual business units play a key role in how open source projects advance at Google.

“We allow the various business units around the company to make the decision on whether it makes sense to open source a given project from a business perspective, because there’s a lot of different reasons why you might open source a project or a piece of code,” he notes. “We’re comfortable with allowing projects to take the approach that works for them given their goals. We play more of a role of facilitating and advising.”

The first guide lays out recommendations for how to include stakeholders ranging from Legal to Engineering in the maintenance of a program office. It also delves into the importance of setting clear program policies and observing compliance guidelines.

“Having a well-defined policy in place, that’s great, but it’s got to be a well-defined minimal policy,” said Jeff Mcaffer, director of the Open Source Programs Office at Microsoft, who was interviewed for the first guide. “Otherwise you get lawyers, security folks, business folks, all piling in their concerns and constraints. Soon you end up with a straitjacket full of policy that basically means that nobody can do anything.”

These free guides are extremely valuable for any organization setting up an open source program. Notably, the guides were not produced in a vacuum. Far from it. The advice you will find in them grew organically out of many interviews with some of the world’s leading open source experts. We strongly encourage you to check out the guides, and stay tuned to this space for more articles in this series.

One of a company’s first challenges when starting an open source compliance program is to find exactly which open source software is already in use and under which licenses it is available.

This initial auditing process is often described as establishing a clean compliance baseline for your product or software portfolio. This is an intensive activity over a period of time that can extend for months, depending on how soon you started the compliance activities in parallel to the development activities.

Below are some recommendations, based on The Linux Foundation’s e-book Open Source Compliance in the Enterprise, for some of the best ways to achieve initial license compliance.

4 Activities to Establish Baseline Compliance

Organizations achieve initial compliance through the following activities:

• Early submission and review of open source usage requests.

• Continuous automated source code inspection based on a predefined interval of time for all source code.

• Continual source code scans, including code received from third-party software providers, to intercept source code that was checked into the code base without a corresponding compliance ticket. Such source code scans can be scheduled to run on a monthly basis, for instance.

• Enforced design and architectural review, in addition to code inspections, to analyze the interactions between open source, proprietary code, and third party software components. Such reviews are mandatory only when a given interaction may invoke license compliance obligations.

Compliance on Future Revisions

If a company fails to establish baseline compliance, it is almost guaranteed that future revisions of the same product (or other products built using the initial baseline) will suffer from compliance issues. To guard against such scenarios, companies should consider establishing other elements of a complete open source management program, including the following:

• Offer simple but enforced policies and lightweight processes.

• Include compliance checkpoints as part of the software development process as it moves from concept into shipping

a product or software stack. Ideally, with every development milestone, you can incorporate a corresponding compliance milestone, ensuring that all software components used in the build have parallel and approved compliance tickets.

• Ensure availability of a dedicated compliance team.

• Utilize tools and automation to support efficient processing of compliance tickets.

There are several challenges in maintaining open source compliance, similar to those faced when establishing baseline compliance. In fact, many of the steps are identical, but on a smaller, incremental scale. We’ll cover recommendations for maintaining compliance in the next article in this series.

Open source compliance

Read the other articles in this series:

The 7 Elements of an Open Source Management Program: Strategy and Process

The 7 Elements of an Open Source Management Program: Teams and Tools

How and Why to do Open Source Compliance Training at Your Company

Basic Rules to Streamline Open Source Compliance For Software Development

How to Raise Awareness of Your Company’s Open Source License Compliance