CoreOS has gained notoriety over the past few years as the creator of a new Linux distribution designed for massive, Google-scale server deployments. The company’s star has risen along with the popularity of Linux containers — a key component of CoreOS — and their open source components are being widely incorporated by companies on the bleeding edge of distributed computing.
This week, for example, CoreOS released etcd 2.0, the first major stable release of this key open source component in distributed systems run by hundreds of companies and projects including Kubernetes, Apache Mesos, and Cloud Foundry. And less than a week earlier, CoreOS released new versions of Rocket and its App Container (appc) specification, which are now both in version 0.2.0.
“There’s a lot of hope for a new world of how you run infrastructure, which is part containers and part distributed systems,” said Alex Polvi, co-founder and CEO of CoreOS. “It’s definitely a new way to run infrastructure and that new way will fundamentally impact how companies do it today versus how they will do it in the future.”
This new approach to enterprise IT is creating a new, as yet undefined, role in the industry, which Polvi calls “Distributed DevOps.”
Polvi will discuss how containers and distributed computing are changing enterprise IT and give an overview of the current key open source projects and companies involved in the transformation at the Linux Foundation’s invitation-only Collaboration Summit to be held Feb. 18-20 in Santa Rosa. (Request an invitation.)
In this Q&A preview he talks containers and enterprise IT, compares the CoreOS container runtime Rocket to Docker, describes the role open source plays in CoreOS’s commercial strategy, and gives us a preview of his CollabSummit keynote.
Linux.com: What about the computing landscape prompted you to start CoreOS? What kinds of hurdles did you hope to overcome?
Alex Polvi: Back in January 2013 when Brandon (Philips, co-founder) left his job and my company was acquired, we were looking around for a project with a meaningful mission and big commercial impact. We wanted to see what we could do to fundamentally improve the security of the Internet.
The key insight in our thesis is that all security boils down to the ability to update software. You can make software hardened and more secure but no software is perfect. We said, let’s build a server that can automatically update itself. That’s very different than the way people think about servers now. If this works, we thought we could unlock a lot of of value, that value being around security, reliability, performance, really everything you get from running the latest version of software.
Everything we do boils down to this mission, though you might think of us as a container OS. We care a lot about containers, but it is because of our ability to help you run your servers in this way. To update automatically, you have to separate your app from the base OS; you can’t tie it in together. The way we do that is with containers. We package the app as a self-contained unit and that lets us update the base OS without breaking the application.
The past year has been a wild ride for Linux containers. How has the rise of this technology changed enterprise IT?
Polvi: There’s a lot of excitement and hope for a new future and people are starting to invest in new platforms around it. But we haven’t seen people going into production in big ways outside of the existing service providers that have been using it before Docker.
The point is, I think there’s a lot of hope for a new world of how you run infrastructure, which is part containers and part distributed systems. There’s a lot of distributed system themes in this emerging style of infrastructure. Things like Kubernetes and Mesos, which kind of play hand-in-hand with the container ecosystem and that’s because it’s a key component.
It’s definitely a new way to run infrastructure and that new way will fundamentally impact how companies do it today versus how they will do it in the future.
Will it be more secure?
Polvi: I mean, if they’re using our products (laughs). I don’t say that to be overconfident I say that because we are sincerely focused on securing the backend of the Internet. If we don’t deliver on that mission, we fail as a company. We sincerely are putting everything we can into security, going beyond just turning it on and making it easy to use. More and more companies are recognizing a lot of the different security technologies are very difficult to use. It’s time to get serious about security.
Going back to how containers have changed enterprise IT, one way that it’s changed is that companies are rethinking everything, right?
Polvi: Yes, it’s really the shift from what I call traditional “IT system administration” to “distributed system administration.” It could be a new industry term similar to “DevOps” for system administration. To me, distributed system administration means part containers and part distributed system methodology. It’s a completely different way but we’ve seen it emerge with Googlein its server infrastructure. Now these models are being applied outside of Google.
When you say distributed system administration, do you mean web-scale IT, essentially?
Polvi: Yes. There aren’t really industry-wide terms that have been adopted yet. It’s like how Google runs its servers for everybody. The problem is that web-scale implies that only large companies need to do it, however this model can be applied to all sizes of companies and all kinds of service climates.
What do you see happening with containers this year?
Polvi: I think we’ll get our first truly production-ready container ready on time between Docker maturing and the technologies we’re creating, as well as Canonical and other folks that are building container stuff. I think you’ll see easy-to-use containers in 2015. That’s my prediction.
What about Rocket? What is it and why is it necessary?
Polvi: Early on in the Docker story (which is still quickly developing,) it got built into a lot of existing platforms so it was a great component for a company building a platform to add containers to it -like Amazon Web Services or Google Cloud, or Mesos. Kubernetes was built as a platform specifically around containers.
What happened more recently is that Docker started to become a platform in and of itself so it will compete with existing platforms. And that’s fine. I understand if they want to build a platform as a company, that makes a lot of sense as a business. The issue is, we still need that simple component to exist for building platforms.
We want to help people with that existing platform to use containers more easily. And so Rocket is designed to be that original, simple component that Docker had, which is a tool for adding containers to your existing platform.
While we were considering the components that are important in the design of a container, we focused on fixing some security issues. The Docker architecture causes some security things that can be very hard to work with. So think of Rocket as original Docker, with cleaned-up security.
When Docker was released, the really cool thing about it was this idea of app portability. For the first time, we have an app packaging format that runs in different environments. Docker is in Amazon and Google. They both can run a software app consistently and that’s great. That’s awesome. For the first time, we got open portability between cloud platforms.
The problem is, Docker did not write down the specifications of what those formats are so that third-party tools could be implemented around it. Essentially, you have to run Docker to use Docker and we believe in a more open model.
In the App Container spec (appc spec) we specify what a container is and what the runtime a container should get is, and then different tools can be built on it to run app containers. We built a command-line tool called Rocket, which is our app container runtime. It is the first implementation of an App Container, and more have followed since it was introduced in December of last year.
I feel that most of what I’ve read about Rocket is about the security and not about the portability.
Polvi: The three value propositions are about security, composability, and open standards (and open standards relate to App Containers). Those are really being developed as independent things. There’s a GitHub repository which is GitHub/Appc, which is app container specification, then there’s Rocket under CoreOS, which is our implementation of App Container. We’re trying to have App Container be the HTTP of containers. Nobody owns HTTP, it’s just used by everything.
What is your commercialization approach?
Polvi: We have two distinct ways that we bring things to market.
First are open source components which are purely open source. This is an open core thing, so there’s no enterprise version. All of our open source components we want to be free to use and embeddable in people’s projects. Our gift to the world is building those things. We build them because we’ve identified them as white space in the world, to be able to run an infrastructure in this new way.
On the commercial side, we sell commercial software products that use our open source components. We sell bold products that help you to run infrastructure in this new way that we haven’t put an industry term on.
One offering is CoreUpdate, which helps you do “rolling upgrades” on your server so you can focus on building applications. The open source version is available if you want to build your own, but in the commercial version we package the best parts of open source for you and take care of the rest via what we call Managed Linux . This includes a foundation managed by CoreOS, automatic security updates, a package manager for containers and professional support.
Then there’s our Enterprise Registry, which is a software registry that companies can use to share and download their docker containers and run it in their own environment. Again, you have the option of an open source version, but we built a commercial version that companies can buy if they want, especially if they want to run on-premise. We will continue to add products to our product line in a similar vein, building enterprise-ready versions for companies to use if they don’t want to piece it all together from open source.
How do Linux and open source fit with the ethos of CoreOS?
Polvi: Linux and open source is at our core. Brandon and I first met at the open source lab at Oregon State, helping run those big OS projects: Kernel.org, Drupal.org, Apache.org. I went to Mozilla really early on in the Mozilla Foundation and started my own company from there. Brandon was a Linux kernel developer.
Everything we do is very open source friendly. We contribute very regularly to the upstream linux kernel and send our patches back up to mainline kernel. We run mainline; we don’t patch it because we’ve gotten all the patches we need in mainline. We are through and through open source folks. Now, we want to be an independent company to work on open source and fill in the white space, while having commercialized parts of our software in order to support that independence and innovation.
Oh, and by the way, you’re actually building something that’s going to change the internet.
Polvi: I hope so! That’s the dream. If we can work on open source technology with a great team of people and build an independent business around it, what else is there?
What will you cover in your Collaboration Summit keynote next month?
Polvi: The main thing that I wanted to cover is everything that’s emerging (in infrastructure) and what’s different between our project and others. I’ll go over the different open source projects and how they’re being commercialized as it relates to containers and distributed systems, as well as some predictions on how it plays out. I want to talk about the difference between Kubernetes, Docker and Mesos; the Docker platform and Docker containers, and Rocket and the work that we’re doing. I will give a lay of the land of everything that’s moving right now in containers because we’re right in the thick of it. It’s a very interesting balance of open source and commercial endeavors. I think it would be good to update everybody.