Posts

Open Source Summit EU

Going to Open Source Summit? Check out some featured conference presentations and activities below.

Going to Open Source Summit EU in Prague? While you’re there, be sure stop by The Linux Foundation training booth for fun giveaways and a chance to win one of three Raspberry Pi kits.

Giveaways include The Linux Foundation branded webcam covers, The Linux Foundation projects’ stickers, Tux stickers, Linux.com stickers, as well as free ebooks: The SysAdmin’s Essential Guide to Linux Workstation Security, Practical GPL Compliance, and A Guide to Understanding OPNFV & NFV.

You can also enter the raffle for a chance to win a Raspberry Pi Kit. There will be 3 raffle winners: names will be drawn and prizes will be mailed on Nov. 2.

And, be sure to check out some featured conference presentations below, including how to deploy Kubernetes native applications, deploying and scaling microservices, opportunities for inclusion and collaboration, and how to build your open source career.

Session Highlights

  • Love What You Do, Everyday! – Zaheda Bhorat, Amazon Web Services
  • Detecting Performance Regressions In The Linux Kernel – Jan Kara, SUSE
  • Highway to Helm: Deploying Kubernetes Native Applications – Michelle Noorali, Microsoft
  • Deploying and Scaling Microservices with Docker and Kubernetes – Jérôme Petazzoni, Docker
  • printk() – The Most Useful Tool is Now Showing its Age – Steven Rostedt, VMWare
  • Every Day Opportunities for Inclusion and Collaboration – Nithya Ruff, Comcast

Activities

  • Technical Showcase
  • Real-Time Summit
  • Free Day with Prague tour from local students
  • KVM Forum
  • FOSSology – Hands On Training
  • Tracing Summit

The Cloud Native Computing Foundation will also a have booth at OSSEU. Get your pass to Open Source Summit Europe and stop by to learn more! Use discount OSSEULFM20 code for 20% off your all-access attendee pass.

Check out the full list of co-located events on the website and register now.

Testing is especially important in modern distributed software systems. Learn more at the upcoming APIStrat conference.

As developers, we often hear that tests are important. Automated testing minimizes the number of bugs released to production, helps prevent regression, improves code quality, supplements documentation, and makes code reviews easier. In short, tests save businesses money by increasing system uptime and keeping developers working on new features instead of fighting fires. While software testing has been around for about as long as software has, I would argue that testing is especially important (and unfortunately more challenging) in modern distributed software systems.

Distributed software” refers to any software that is not run entirely on one computer. Almost all web applications are distributed software as they rely on applications on other servers (eg: remote data stores, internal REST APIs, third-party APIs, content delivery networks, etc.), and most mobile and desktop applications are as well. Distributed software presents new challenges and requires a thoughtful approach to testing. This list includes just some of the reasons that testing is crucial for distributed software:

1. Third Party APIs Can Change Unexpectedly

We would like to think that every REST API we use will adhere to some form of versioning, but this doesn’t always happen. Sometimes APIs break when a maintainer fixes a bug, sometimes breaking changes are overlooked, and sometimes the API just isn’t mature or stable yet. With more companies releasing public APIs, we’re bound to see the number of accidentally breaking releases rise, and tests are a great way to prevent those breaking changes from affecting our applications.

2. Internal API Changes can Affect Your App in Unexpected Ways

Even more commonly, breaking API changes come from within our own organization. For the past few years, I’ve been working with startups where the business requirements change almost as fast as we can implement them, so our internal APIs are rarely stable and sometimes the documentation gets outdated. Slowing down, improving communication between team-members, and writing tests for our internal APIs has helped.

3. Remotely Distributed Open Source Packages are More Popular Than Ever

78% of companies are now running on some form of open source software. This has helped the speed and ease of developing software to increase exponentially, but blindly relying on open source packages has bitten plenty of developers as well (see the left-pad incident of 2016). Once again, we hope that open source packages use semantic versioning, but it’s impossible to guarantee this. Testing the boundaries between packages and our software is one way to help improve reliability.

4. Network Connections Aren’t Perfect

In many server-to-server cases, network connections are pretty reliable, but when you start serving up data to a browser or mobile client via an API, it gets much harder to guarantee a connection. In either case, you should have a plan for failure: Does your app break? Throw an error? Retry gracefully? Adding tests that simulate a bad network connection can be a huge help in minimizing poor user experiences or data loss.

5. Siloed Teams can Lead to Communication Gaps

One of the advantages to distributed systems is that a team can be assigned to each component. This allows each team to become an expert on just one part of the system, enabling the scaling of software organizations like we see at Amazon. The downside to these siloed teams is that communication becomes more difficult, but a good test suite, thorough documentation, and self-documenting APIs can help minimize these gaps.

How Do We Test Distributed Systems?

Distributed software has become more popular as the cost of cloud computing has gone down and network connections have become more reliable. While distributed systems offer unique advantages for scaling and cost savings, they introduce new challenges for testing.

Borrowing from some of Martin Fowler’s ideas on testing microservices and my own experience building layered test plans, I’ll be presenting a strategy for testing distributed systems at this year’s API Strategy & Practice Conference. If you’re interested in learning more about the topic of testing distributed software, or you have questions, you can find me at the conference, or anytime on Twitter.

Learn more at APIStrategy and Practice Conference.

Reuben Paul, co-founder of CyberShaolin, will speak at Open Source Summit in Prague, highlighting the importance of cybersecurity awareness for kids.

Reuben Paul is not the only kid who plays video games, but his fascination with games and computers set him on a unique journey of curiosity that led to an early interest in cybersecurity education and advocacy and the creation of CyberShaolin, an organization that helps children understand the threat of cyberattacks. Paul, who is now 11 years old, will present a keynote talk at Open Source Summit in Prague, sharing his experiences and highlighting insecurities in toys, devices, and other technologies in daily use.

Reuben Paul, co-founder of CyberShaolin

We interviewed Paul to hear the story of his journey and to discuss CyberShaolin and its mission to educate, equip, and empower kids (and their parents) with knowledge of cybersecurity dangers and defenses.  

Linux.com: When did your fascination with computers start?
Reuben Paul: My fascination with computers started with video games. I like mobile phone games as well as console video games. When I was about 5 years old (I think), I was playing the “Asphalt” racing game by Gameloft on my phone. It was a simple but fun game. I had to touch on the right side of the phone to go fast and touch the left side of the phone to slow down. I asked my dad, “How does the game know where I touch?”

He researched and found out that the phone screen was an xy coordinate system and so he told me that if the x value was greater than half the width of the phone screen, then it was a touch on the right side. Otherwise, it was a touch on the left side. To help me better understand how this worked, he gave me the equation to graph a straight line, which was y = mx + b and asked, “Can you find the y value for each x value?” After about 30 minutes, I calculated the y value for each of the x values he gave me.

“When my dad realized that I was able to learn some fundamental logics of programming, he introduced me to Scratch and I wrote my first game called “Big Fish eats Small Fish” using the x and y values of the mouse pointer in the game. Then I just kept falling in love with computers.Paul, who is now 11 years old, will present a keynote talk at Open Source Summit in Prague, sharing his experiences and highlighting insecurities in toys, devices, and other technologies in daily use.

Linux.com: What got you interested in cybersecurity?
Paul: My dad, Mano Paul, used to train his business clients on cybersecurity. Whenever he worked from his home office, I would listen to his phone conversations. By the time I was 6 years old, I knew about things like the Internet, firewalls, and the cloud. When my dad realized I had the interest and the potential for learning, he started teaching me security topics like social engineering techniques, cloning websites, man-in-the-middle attack techniques, hacking mobile apps, and more. The first time I got a meterpreter shell from a test target machine, I felt like Peter Parker who had just discovered his Spiderman abilities.

Linux.com: How and why did you start CyberShaolin?
Paul: When I was 8 years old, I gave my first talk on “InfoSec from the mouth of babes (or an 8 year old)” in DerbyCon. It was in September of 2014. After that conference, I received several invitations and before the end of 2014, I had keynoted at three other conferences.

So, when kids started hearing me speak at these different conferences, they started writing to me and asking me to teach them. I told my parents that I wanted to teach other kids, and they asked me how. I said, “Maybe I can make some videos and publish them on channels like YouTube.” They asked me if I wanted to charge for my videos, and I said “No.” I want my videos to be free and accessible to any child anywhere in the world. This is how CyberShaolin was created.

Linux.com: What’s the goal of CyberShaolin?
Paul: CyberShaolin is the non-profit organization that my parents helped me found. Its mission is to educate, equip, and empower kids (and their parents) with knowledge of cybersecurity dangers and defenses, using videos and other training material that I develop in my spare time from school, along with kung fu, gymnastics, swimming, inline hockey, piano, and drums. I have published about a dozen videos so far on the www.CyberShaolin.org website and plan to develop more. I would also like to make games and comics to support security learning.

CyberShaolin comes from two words: Cyber and Shaolin. The word cyber is of course from technology. Shaolin comes from the kung fu martial art form in which my dad and are I are both second degree black belt holders. In kung fu, we have belts to show our progress of knowledge, and you can think of CyberShaolin like digital kung fu where kids can become Cyber Black Belts, after learning and taking tests on our website.

Linux.com: How important do you think is it for children to understand cybersecurity?
Paul: We are living in a time when technology and devices are not only in our homes but also in our schools and pretty much any place you go. The world is also getting very connected with the Internet of Things, which can easily become the Internet of Threats. Children are one of the main users of these technologies and devices.  Unfortunately, these devices and apps on these devices are not very secure and can cause serious problems to children and families. For example, I recently (in May 2017) demonstrated how I could hack into a smart toy teddy bear and turn it into a remote spying device.
Children are also the next generation. If they are not aware and trained in cybersecurity, then the future (our future) will not be very good. 

Linux.com: How does the project help children?
Paul:As I mentioned before, CyberShaolin’s mission is to educate, equip, and empower kids (and their parents) with knowledge of cybersecurity dangers and defenses.

As kids are educated about cybersecurity dangers like cyber bullying, man-in-the-middle, phishing, privacy, online threats, mobile threats, etc., they will be equipped with knowledge and skills, which will empower them to make cyber-wise decisions and stay safe and secure in cyberspace.
And, just as I would never use my kung fu skills to harm someone, I expect all CyberShaolin graduates to use their cyber kung fu skills to create a secure future, for the good of humanity.

The Tools for Managing Open Source Programs guide provides an exhaustive collection of categorized tools that any open source program can benefit from.

Is your organization looking to build out an open source program? If so, you’re not alone, but not every organization has a holistic sense of the available tools that can help create a healthy program. A simple charter document and a few spreadsheets for tracking projects won’t cut it anymore in managing a truly robust open source program. That’s where the new Tools for Managing Open Source Programs guide comes in. It can help any organization launch and maintain a thriving open source program.

“If you have more than 100 code repositories or 100 people that you’re trying to manage, you really can’t have someone doing it manually with spreadsheets anymore,” notes Jeff McAffer, Director of the Open Source Programs Office at Microsoft, in the guide. “Obviously, people still do it that way. But it starts to become ad-hoc and laborious. That’s where tools come into play. They allow you to scale.”

While launching and maintaining an open source program does require dedicated, task-specific tools, it is a mistake to assume that your organization must necessarily build its own tools from the ground up.

“Regarding existing tools and systems, my hope is that we’re quickly getting to a point where a company’s open source program office should not need to create any tools or technologies on their own,” said McAffer. “They should be able to find and use existing open source tools which can be used to manage their open source programs.”

Categorized Tools

The Tools for Managing Open Source Programs guide provides an exhaustive collection of categorized tools that any open source program can benefit from. These include Source Code Scanning and License Compliance tools, Bug Tracking tools, Release Management tools, and more. Are you familiar with FOSSology? It’s a Linux Foundation project that functions as an open source license compliance software toolkit capable of running license, copyright and export control scans from the command line. Have you heard of Docker Hub? It’s a cloud-based registry service that allows users to link to code repositories and build and test their images. These and many, many more useful tools are linked to and explained in the free guide.

Do you know how to answer questions like these?

  • How are your project APIs documented?
  • Have you laid out a Contributor Licensing Agreement that everyone can use?
  • Have you picked the right license for your project?

Various tools can help you determine the right answers to these questions, and the Tools for Managing Open Source Programs guide is a great way to surface them.

Methodologies

It’s important to understand that using open source for business strategy requires its own methodologies and processes which are very different than those needed when using and releasing proprietary software. As the guide notes:

“Nobody said it was going to be simple to move your company into the world of open source. But plenty of other companies, including giants like Microsoft and Google have done this before you and have provided detailed road maps, code, suggestions, and more to make your own journey easier. The creation of an open source program office and the selection of a package of critical tools to get your efforts started are within your grasp. By collaborating on open source projects and inviting others to collaborate with you, your company can gain immeasurable benefits and drive its progress forward with energy and innovation.”

The Tools for Managing Open Source Programs guide is one of a new collection of guides from The Linux Foundation and The TODO Group that are all extremely valuable for any organization setting up an open source program. The guides are available now to help you run an open source program office where open source is supported, shared, and leveraged. With such an office, organizations can establish and execute on their open source strategies efficiently, with clear terms.

These guides were not produced in a vacuum. Far from it. The advice you will find in them grew organically out of many interviews with some of the world’s leading open source experts. We encourage you to check out the guides and stay tuned for our continuing coverage of them.

Also, don’t miss the first article in this series, on How to Create an Open Source Program, which explores everything from the role of the open source program office to how successful open source programs at companies like Google function.

 

Concepts such as decentralizing strategy, delegating direction, and fierce transparency in communication are part of the backbone of successful open source projects. In my presentation at Open Source Summit EU in Prague, I will explore how these concepts are not only applicable to volunteer-run organizations but can also help growing corporations avoid some of the coordination overhead that often comes with growing teams and organizations.

We’ll look at some of the key aspects of how project members collaborate at The Apache Software Foundation (ASF). After that, we’ll take a closer look at German FinTech company Europace AG, which decided to move toward self-organization two years ago. We’ll highlight parallels between Europace AG’s organizing approaches and those of open source projects.

Let’s start with some of the core values of ASF projects.

Community over Code

One main principle is the concept of “community over code” — which means that without a diverse and healthy team of contributors to a project, there is no project. It puts the team front and center, as highlighted in the Apache project maturity model.

Meritocracy

Another core value to Apache projects is meritocracy — essentially meaning that there is no governance by fiat. There is no way to simply buy influence into projects — you have to invest time to gain influence. This directly translates to frequently given advice for how to get started with any given project: Focus on the things you are using yourself and fix what bothers you most essentially following a scratch your own itch kind of model. There is one level of indirection here: Committers and contributors can be paid either by their employers or as part of a consulting gig to increase their motivation to work on the topics that you are interested in.

Project independence

Be aware though that there is a strong principle of people participating as individuals on projects. That level of independence means that within an Apache project, there is no way to assign tasks to project members. Apache projects by design have a very flat organization with barely any hierarchy: Titles like Project Management Committee Chair (PMC Chair) are famous for coming with additional responsibility but no entitlement to task assignment. That means projects need a process for aligning people with potentially very diverse interests and itches to scratch. A key ingredient to coming up with workable decisions is making project goals and needs public and transparent — both in terms of asking for help and in showing appreciation and rewarding contributions that help the project. Another one can be found in an approach to integrating differing opinions and arguments in a “Yes, and” instead of a “Yes, but” fashion. This plays together very closely with the next concept.

Full transparency in open source projects

“What didn’t happen on the mailing list, didn’t happen” is one of the mantras for those involved with Apache projects. A high level of transparency is what makes Apache projects so easy to participate in across boundaries — whether they are geographic, corporate, or other. At Apache, mailing lists are treated as the point of reference for any decisions made. Thus, there is full transparency into the historic record of all open source projects.

Of course, with that entirely open and transparent model comes responsibility: A lot of decisions can be taken in the open. People tend to show better behavior when under public scrutiny. However, discussions involving interpersonal issues are best kept out of public sight. This is particularly important in a digital age where deleting statements made online and mirrored worldwide is pretty much impossible.

Project autonomy

Technologically, Apache projects are very diverse. However, when unified under one organization with a common focus on community, meritocracy, and communication patterns, there is a lot of freedom and decentralized decision making.

One important piece in growing the ASF as a whole was giving autonomy to each project. Still there are a couple topics that each project has to deal with: It’s great having one central entity answering questions on, for example. trademarks, licensing, or software patents. However, these entities controlling each of the 300 or so top level open source projects won’t scale. Instead, Foundation-level services serve as guidelines for Apache open source projects to manage their own activities in these areas. As a result, each project is supposed to have some people knowledgeable in those topics.

Open source patterns in the enterprise

In the past decade, people in the open source space have successfully brought several of these concepts to their respective employers — either in the form of open development, inner source, or open organizations. Meanwhile, ideas of transforming traditional hierarchical organizations into self-organized, open organizations found their way into concepts like sociocracy and holacracy.

Europace AG decided to move toward self-organization roughly two years ago. Looking at what was established during these two years, we see quite a few similarities to how nonprofits such as the Apache Software Foundation and The Linux Foundation work with open source projects:

Decentralization and autonomy

Europace AG is a tech company experienced with XP and Scrum, and in 2015, they began a journey toward self-organization, looking into holacracy and sociocracy frameworks. Organizationally, there are roughly 200 people split into four fairly autonomous technical units, each one of which is responsible for one business product of the company. Units are autonomous in how they self-organize: Some chose fairly standard Scrum-based software development in iterations and standard rituals. Others went for sociocratic or holacratic models. By organizing in circles, employees are given more influence over how decisions are made that affect their daily work. Backing for this level of autonomy can be found in the company’s own principles, where self-organization and decentralization are outlined as one of the four important values and are actively supported by the board.

This is not unlike Apache, where open source projects operate independently and autonomously in how they develop their software and self-organize (that is, provided they make sure that project governance remains independent of one single player, security issues are being dealt with, and legalities like licensing, patent issues, and trademarks are taken care of).

Transparent decision making

Everyone working for and with Europace AG is located in one timezone, and Europace AG itself has only one site. Nonetheless, there are good reasons to look at how decisions are made in distributed open source projects: Establishing a remote friendly communication model can help support employees with family, and deal with long commute distances. Apart from these obvious benefits, there are a few effects that aren’t quite obvious:

While an asynchronous communication model won’t lead to decisions happening faster, it can help with reducing the amount of interruptions caused by face-to-face meetings: By sharing a proposal before the actual meeting happens and providing a means to exchange ideas around that proposal can help with understanding people’s opinions, issues and suggestions with respect to the proposal. It can help with making that understanding transparent to everyone interested in the topic up front. Combined with a “Yes, and” philosophy, this can help form consent in the team quickly.

Organizational Groups

Splitting roughly 200 employees into just four units creates groups that are bigger than typical “2 pizza sized teams” often deemed ideal for software development teams. As a result, each unit had to figure out how to create sub-groups that can work well as teams.

A common approach was to create cross-functional feature teams, comprising both software developers and people familiar with financial domain (and people with backgrounds in data analytics, UX, etc. as needed). Those are called feature teams in anticipation of the fact that they aren’t necessarily particularly stable: Each week, there is a check point people can use to decide to change feature teams depending on where they are needed most. Decisions related to governance as well as product strategic aspects are delegated to the team itself. As a result, team members need a good understanding of the current product direction in terms of features and usage.

For tasks and topics that tend to fall under the general topic of “organizational operations” or “decision making with dependencies,” people organize themselves in circles, which are essentially groups of people interested in participating in making a decision related to a specific topic. Despite a lot of communication still happening face to face, some circles also set up a Slack channel that is open for all to join and participate. This level of transparency is crucial in elevating trust people have in those decisions.

Each of these circles can come with dedicated roles: Secretary being the one taking over meeting minute creation, a Lead Link making sure that “things will actually get done” and communicated to the people needing the respective information. While people can volunteer for these roles, at the end of the day these roles get filled through nomination by the team. This is not unlike Apache projects where new committers, PMC members, and even Foundation members are elected after nomination. On one hand, this kind of model shows a certain amount of appreciation for the work these people are doing. On the other, it helps with promoting people into these roles who according to their own self-assessment might not have confidence to take on the role.

Put the human first 

Much like Apache puts community development front and center with its open source projects, Europace AG puts team collaboration into a central position. This means that each employee is responsible for raising issues with team climate, and they are supported by a team of professional coaches, moderators, and mediators with a deep understanding of human communication.

Conclusion

When moving toward self-organization, corporations can learn a lot from how open source projects organize themselves. Europace AG is on the journey toward moving more power away from traditional formal structures, making it available for the people in purpose-driven, circle-based organization. It will be interesting to watch how Open Source project management, governance, and communication patterns can be applied within corporate contexts.

Isabel Drost-Fromm is Open Source Strategist at Europace AG Germany. She is a member of the Apache Software Foundation, co-founder of Apache Mahout and has mentored several incubating projects. Isabel is interested in all things FOSS, search and text mining with a decent machine learning background.