Posts

More than 30 members join new Ceph Foundation to manage the massive growth in data and information generated from cloud, container and AI applications 

CEPH DAY BERLIN – Nov. 12, 2018 – The Linux Foundation, the nonprofit organization enabling mass innovation through open source, today announces over 30 global technology leaders are forming a new foundation to support the Ceph open source project community. The Ceph project develops a unified distributed storage system providing applications with object, block, and file system interfaces.

Founding Premier members of Ceph Foundation include Amihan, Canonical, China Mobile, DigitalOcean, Intel, OVH, ProphetStor Data Services, Red Hat, SoftIron, SUSE, Western Digital, XSKY Data Technology, and ZTE. The Ceph Foundation will organize and distribute financial contributions in a coordinated, vendor-neutral fashion for immediate community benefit. This will help galvanize rapid adoption, training and in-person collaboration across the Ceph ecosystem.

“Ceph has a long track record of success when it comes to helping organizations with effectively managing high growth and expanding data storage demands,” said Jim Zemlin, Executive Director of the Linux Foundation. “Under the Linux Foundation, the Ceph Foundation will be able to harness investments from a much broader group to help support the infrastructure needed to continue the success and stability of the Ceph ecosystem.”

Ceph is used by cloud providers and enterprises around the world, including financial institutions (Bloomberg, Fidelity), cloud service providers (Rackspace, Linode), academic and government institutions (Massachusetts Open Cloud), telecommunications infrastructure providers (Deutsche Telekom), auto manufacturers (BMW), software solution providers (SAP, Salesforce), and many more.

Ceph is also used by Rook, a Cloud Native Computing Foundation project that brings seamless provisioning of file, block and object storage services into the Kubernetes environment, running the Ceph storage infrastructure in containers alongside applications that are consuming that storage.

Efficient, agile, and massively scalable, Ceph significantly lowers the cost of storing enterprise information in the private cloud and provides high availability to any object, file, and block data. Unstructured data makes up 80% and more of enterprise data, is growing at the rate of 55% to 65% per year, and is common with rich-media, predictive analytics, sensors, social networks, and satellite imagery.[1] Block and file storage are critical to any IT infrastructure organization and are important components of infrastructure platforms like OpenStack and Kubernetes. According to recent user surveys, roughly two-thirds of OpenStack clouds use Ceph block storage.

The growth of new cloud, container and artificial intelligence/machine learning applications are driving increased use of Ceph. For example, Ceph combined with analytics and machine learning enables enterprises to comb through mass amounts of unstructured data to quickly spot patterns with customer behavior, online customer conversations and potential noncompliance scenarios.

The Ceph Foundation is the successor framework to the Ceph Advisory Board, which was formed in 2015. Get involved in the Ceph community at https://ceph.com/get-involved/ or learn more about membership at https://ceph.com/foundation.

Project Creator and Leadership Quotes

“A guiding vision for Ceph is to be the state of the art for reliable, scale-out storage, and to do so with 100% open source,” said Sage Weil, Ceph co-creator, project leader, and chief architect at Red Hat for Ceph. “While early public cloud providers popularized self-service storage infrastructure, Ceph brings the same set of capabilities to service providers, enterprises, and individuals alike, with the power of a robust development and user community to drive future innovation in the storage space. Today’s launch of the Ceph Foundation is a testament to the strength of a diverse open source community coming together to address the explosive growth in data storage and services.”

“Ceph was designed and built for scalability, initially with supercomputers and later with cloud infrastructure in mind. A key design premise was that the storage system needs to provide a highly reliable and available service in a dynamic and increasingly heterogeneous hardware environment where everything can potentially fail,” said Carlos Maltzahn of University of California, Santa Cruz, a co-founder of the research project that first created Ceph over a decade ago.

Cephalocon and KubeCon + CloudNativeCon 2019

The Ceph Foundation plans to host the second Cephalocon conference in Barcelona, Spain from May 19 – 20, 2019, co-located with KubeCon + CloudNativeCon 2019, May 20-23. The first Cephalocon took place in March 2018 in Beijing, China, bringing together 1000 attendees consisting of Ceph vendors, users, and developers from all over the world.

For updates, subscribe to the Ceph project’s announcement mailing list and follow Ceph on Twitter @Ceph.

Founding Members

Founding members of the Ceph Foundation at the Premier level include Amihan, Canonical, China Mobile, DigitalOcean, Intel, OVH, ProphetStor Data Services, Red Hat, SoftIron, SUSE, Western Digital, XSKY Data Technology, and ZTE.

Ceph Foundation General members include Ambedded Technology, Arm, Catalyst Cloud, Croit GmbH, EasyStack, Intelligent Systems Services, Pingan Technology, QCT, Sinorail, and Xiaoju Science Technology.

Associate members include Boston University Information Services and Technology, CERN (European Organization for Nuclear Research), FAS Research Computing – Harvard, Greek Research and Technology Network (GRNET), Monash University, South African Radio Astronomy Observatory (SARAO), Science and Technology Facilities Council (STFC) at UK Research and Innovation (UKRI), and University of California Santa Cruz’s Center for Research in Open Source Software (CROSS).

Supporting Quotes

“As a founding member of the Ceph Foundation, Amihan is committed to contributing its knowledge and expertise in cloud-native infrastructure to the advancement of the Ceph project. Building on Ceph has allowed us to provide our enterprise clients with the most reliable foundation for digital transformation.”

  • Winston Damarillo, executive chairman, Amihan

“Ceph delivers enterprise-grade storage with best-in-class economics. Given the exponential growth we are witnessing in our customers’ data requirements, we recognize it to be a fundamental part of the open source private cloud stack. Our participation in the launch of the Ceph Foundation underlines our commitment to accelerating its adoption, technology base and developer community.”

  • Christian Reis, VP, Data Centre Field Engineering at Canonical

“China Mobile always embraces Open Source and gives back to Open Source. Ceph is the best implementation of Open Source Software Define Storage, and it has been widely used in China Mobile’s data center. We are very happy to join Ceph Foundation and make Ceph more closer to enterprise storage level.”

  • Junwei Liu, vice general manager of cloud computing department, China Mobile (Suzhou) software technology Co., Ltd.

“Open source has been a key part of our business from the start. Ceph is a core component of our fast-growing storage infrastructure and one of the many open source projects we use at DigitalOcean to accelerate innovation and deliver great products for our customers. We are proud to be a founding member of the Ceph Foundation. As a leading cloud provider we believe it is our responsibility to give back to the open source community and support the projects that help developers build amazing apps.”

  • Shiven Ramji, VP of Product, DigitalOcean

“OVH is thrilled to join the newly created Ceph Foundation as a Premier member. Its Open Source values fit in very well with our own DNA, which is marked by openness, innovation and freedom of choice. With Ceph being the de-facto standard Software Defined Storage solution for OpenStack, it was only logical for us to adopt Ceph as part of our Public Cloud block storage. We have been using Ceph for many years and have since contributed numerous patches that went upstream in Ceph, co-hosted the Ceph Day in Warsaw and shared our experience with Ceph at various events. As one of the founding sponsors of the Ceph Foundation, we are now excited to make an even stronger contribution to the Ceph community and help take it to the next level.”

  • Alain Fiocco, EVP Chief Technology Officer, OVH

“The incredible growth of data generated by connected devices, services, and modern software requires an advanced, adaptable storage platform to help gain the greatest value from all this information. Ceph’s open innovation is key for the industry to address these demands. Intel is proud to join the Ceph Foundation and support the redefinition of data storage.”

  • Imad Sousou, Corporate vice president and general manager Open Source Technology Center, Intel Corporation

“Ceph is the future of storage. We at ProphetStor are proud to have contributed our machine learning technology to its community to help bring it to the forefront of the storage industry.”

  • Jeremy Wei, VP of Corporate Business Development, ProphetStor

“Red Hat is honored to be deeply involved as a leader in open source communities as modern enterprises pivot toward an open hybrid cloud model. The creation of the Ceph Foundation brings to bear collaboration across the industry to enable rapid adoption of advanced data services and cloud-native technologies.”

  • Sarangan Rangachari, Vice President and General Manager, Red Hat Storage

“Ceph is quickly becoming the de facto standard for storage at scale. We are proud to be a part of that success and support the new Ceph Foundation. Ceph is a shining example of how open source software can squeeze out inefficiencies in very large markets.”

  • Tim Massey, CEO, SoftIron

“The formation of the Ceph Foundation under the Linux Foundation umbrella reflects the professional growth of the Open Source community and increasing industry participation. The new framework will lead to more community collaboration, outreach, inclusion and market awareness. SUSE is pleased and committed to be a top tier contributor to both the technical community and the foundation to help facilitate innovation in software-defined storage and represent the interests of our customers and partners.”

  • Lars Marowsky-Brée, Distinguished Engineer, SUSE

“As a technology leader enabling cutting-edge technologies with data infrastructure, we understand and believe in open standards, open software, and open ecosystems. The establishment of the Ceph Foundation is an important milestone that we are proud to support.”

  • Allen Samuels, Engineering Fellow at Western Digital, Inc

“With the establishment of Ceph Foundation, Ceph is opening a new era in global development. Together with other members, XSKY will spare no effort to push forward the construction of a broader Ceph ecosystem and explore commercial models for open-source software. We aim to achieve a landing storage approach of high-availability and high-reliability for users in mass data era.”

  • Haomai Wang, CTO, XSKY

“It has been my pleasure to see the growth in popularity of Ceph in China over the last several years, and the power of Ceph of transforming numerous organizations’ IT infrastructure. Open source is a huge opportunity for organizations to work cooperatively to improve software and to benefit from each others’ hard work, and the establishment of Ceph Foundation will definitely accelerate its development and bring us a real future of storage.”

  • Xinyu Chen, General Manager of Telecom Cloud & CoreNet Product, ZTE Corporation

About the Ceph Foundation
Ceph is a unified distributed storage system providing applications object, block, and file system interfaces in a single unified storage cluster—making Ceph flexible, reliable and easy for you to manage. Ceph is built on the Reliable Autonomic Distributed Object Store (RADOS), which provides a highly available and scalable fabric that can either be consumed directly or via higher-level object, block and file services that are built on top. The Ceph Foundation is a directed fund under The Linux Foundation. It provides a neutral home for the Ceph community to collaborate and grow, and is supported by members across multiple industries.

About the Linux Foundation
The Linux Foundation is the organization of choice for the world’s top developers and companies to build ecosystems that accelerate open technology development and industry adoption. Together with the worldwide open source community, it is solving the hardest technology problems by creating the largest shared technology investment in history. Founded in 2000, The Linux Foundation today provides tools, training and events to scale any open source project, which together deliver an economic impact not achievable by any one company. More information can be found at www.linuxfoundation.org.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: https://www.linuxfoundation.org/trademark-usage. Linux is a registered trademark of Linus Torvalds.

Media Contact
Emily Olin
The Linux Foundation
eolin@linuxfoundation.org

[1] Source: https://www.datamation.com/big-data/structured-vs-unstructured-data.html

OpenMessaging

The goal of the OpenMessaging Project is to build out an industry standard, cloud oriented, and vendor neutral open standard for distributed messaging.

Today, the OpenMessaging Project a collaborative project focused on creating a vendor-neutral open standard for distributed messaging announced four new members JD.com, China Mobile Cloud, QingCloud, and Whale Cloud. Current members include Alibaba, DataPipeline, Di Di, Streamlio, WeBank, and Yahoo!.

The acceleration of microservice-based and cloud-based applications has put a growing focus on how data is connected to services, applications, and users. This focus has led to a number of new innovations and new products that support messaging and queueing needs. It has also contributed to increased demands on messaging and queuing solutions, making performance and scalability critical to success, and the need for an open standardization a must.

The goal of the OpenMessaging Project is to build out an industry standard, cloud oriented, and vendor neutral open standard for distributed messaging. More on this project and how to participate here: http://openmessaging.cloud

New Member Supporting Quotes:

“At China Mobile and CMsoft, we have built a MQ proxy system of Apache RocketMQ  to provide a set of producer APIs and consumer APIs. The redundancy of having to hide the differences among the MQs takes so much time and energy out of our team. Given our knowledge in this field, we understand first hand the importance of a messaging communication standard. Having a vendor-neutral and language-independent MQ standard guideline is a big win for many applications. We believe this standard can help and promote the MQ technology that we rely on.” – Henry Hu, Architect at China Mobile and CMsoft.

“As a cloud provider, we offer various messaging services including Apache Kafka, RabbitMQ, and RocketMQ to our customers. More and more people keep asking us what software to use for their messaging requirements as the market is saturated with various open source solutions. This market saturation causes not only a high learning curve, but also a high maintenance cost. An industry open standard, vendor-neutral and language-independent specification for distributed messaging is increasingly important, especially in a cloud era. We look forward to collaborating with the OpenMessaging project to help drive messaging service towards a unified, open standard interface.” – Ray Zhou, Development Director at QingCloud

At the JD Group, JingDong Message Queue (JMQ) has been widely used. However, despite our efforts to be compatible with all kinds of message protocols, we still can’t meet all the requirements. We are planning to open source JMQ, so it can be implemented for OpenMessaging. We see OpenMessaging as a de-facto international open standard for distributed messaging that aims at satisfying the need of modern cloud-native messaging and streaming applications. We sincerely believe that a unified and widely-accepted messaging standard can benefit MQ technology and applications relied on it.” – DeQiang Lin, Messaging Leader at the JingDong Middleware Department

“Currently, message queuing uses proprietary, closed protocols, restricting the ability for different operating systems or programming languages to interact in a heterogeneous set of environments. At Whale Cloud, in order to make it easy for developers to use messaging and streaming services, we’ve worked to eliminate the differences between the different protocols. Giving us insight and knowledge to know that a vendor-neutral and language-independent open specification is badly needed.” – Zheng Tao, Technical Director of Distributed Messaging and Streaming Data Platform at Whale Cloud

Pinterest — “the world’s catalog of ideas” — is built on open source, according to Jon Parise, technical architecture lead and open source program lead at the company. In this interview, Parise explains how adopting open source has changed the company and helped the company’s engineers design software that is more modular, reusable, and well-documented from the outset.

Pinterest also frequently open sources technology they build “as both a way to give back to communities and because it’s the right thing to do,” Parise says. Read on to learn more about how Pinterest uses and contributes to open source.

jon-parise-pinterest.png

Jon Paris

Jon Parise, technical architecture lead and Open Source program lead, Pinterest

Linux.com: What does Pinterest do?

Jon Parise: Pinterest is the world’s catalog of ideas, used by 150 million people every month to discover and do things they love.

Linux.com: How and why do you use Linux and open source?

Parise:  Pinterest is built on open source. From our first lines of Django code over six years ago to the release of Rocksplicator, our latest open source project, we’ve directly benefited from open source projects and the supportive communities surrounding them. In turn, we’ve made significant contributions to the open source technologies we use and have pushed the limits of technologies like HBase. Internally, open source is part of a product cycle. We frequently open source the technologies we build as both a way to give back to communities and because it’s the right thing to do.

Linux.com: Why did you join The Linux Foundation?

Parise: The Linux Foundation is home to Linux, Node.js and other mission critical projects that form the backbone of modern internet services, including Pinterest. Joining the Linux Foundation is great way for established companies like ours to support those communities.

Linux.com: What interesting or innovative trends in your industry are you witnessing and what role do Linux and open source play in them?

Parise:  We’re excited about three big technology trends: machine learning, computer vision and rich media.

Almost all machine learning efforts at Pinterest rely on open source components to some extent. Our data processing pipelines run on Hadoop, Cascading and Scalding. We use TensorFlow, Caffe and XGBoost to train our models, and R and Python are widely used for analyzing the results produced.

Our Visual Search team uses open source libraries for deep learning and computer vision. Caffe, TensorFlow and OpenCV are critical building blocks at Pinterest and allow us to actively publish our algorithmic and implementation findings for building performant visual search systems.

Lastly, our video and image platform leverages libraries like FFmpeg, GraphicsMagick, and ExoPlayer to power efficient media processing and playback.

Linux.com: How has participating in open source communities changed your company?

Parise:  As we’ve grown and are better resourced, we can now be a better contributor to open source by designing our software to be modular, reusable and well-documented from the beginning. In addition to making it easier for us to open source parts of our software down the road, that mindset benefits us in our day-to-day work as engineers inside the company, too.

We also strive to be additive to the greater open source community by filing bug reports, contributing patches, publishing papers, hosting tech talks and meetups, and speaking at conferences. Those are important things everyone can do, both as individuals and as employees of companies that benefit from open source.

Linux.com: How do you participate in the TODO Group and what have you learned from the experience?

Parise:  It’s early for us as members, but we’re excited to participate in a group of like-minded companies committed to open source. In particular, TODO focuses on how companies can run effective open source programs both internally and externally. Pinterest has already released over 30 open source projects, and with untold more to come, the benefit of TODO’s collective experience has already proven invaluable.

Learn more about Linux Foundation corporate membership and see a full list of members at https://www.linuxfoundation.org/members/join.