A potentially overlooked Kernel Summit outcome

Corbet's picture

Every year, the Linux Kernel Summit brings together about 100 core kernel developers and subsystem maintainers to discuss issues of importance to the development community as a whole.  The 2014 Summit, held August 18-20 in Chicago (just before LinuxCon North America) covered a wide range of topics and made a number of decisions.  But there is one outcome from this event that merits especially wide attention: the development community is getting better at what it does.

Regressions are the bane of software developers and users everywhere.  If a software upgrade breaks a system, or even just makes it run more slowly, lots of time can be lost making things work again and trust in the system as a whole is affected.  Regressions have long been a concern in the kernel community, so it is not surprising that they are often discussed at the Kernel Summit.

This year's Summit brought an interesting surprise, though.  Facebook developer Chris Mason talked about that organization's experience with moving to the 3.10 kernel on its production servers.  Such moves have often brought to light various regressions that must be addressed.  But the number of problems in 3.10, Chris said, was far lower than expected.  Indeed, it was, he said, the easiest kernel move ever.  Perhaps even better, an experimental move to 3.16 — a very new kernel release — also turned up very few problems.

Dave Jones (from Red Hat), instead, talked about his work with defect reports from Coverity's static analysis system.  Every reported problem may indicate a bug or security issue, so they need to be taken seriously.  In the year that Dave has been working with these reports, the reported defect density has dropped considerably.  And, happily, new development kernels are not adding new problems at anything near the rate with which they are being removed.  We have, it seems, gotten better at avoiding adding bugs to the kernel.

Why is the kernel getting better in this way?  Certainly much of the credit belongs to kernel developers who are putting more effort into the quality of their code.  But it also belongs with the companies that have been putting significant resources into proactive kernel testing.  With the testing resources that are now in place, many problems can be found long before they can affect users; indeed, many are found before the code in question makes it into the mainline kernel at all.  The end result is that a development community that was already known for releasing high-quality kernels is getting even better; that is good news for Linux users everywhere.