The explosive growth of data from Internet-connected devices has led to massive demand for storage and analysis tools in computing. Treasure Data aims to serve both of these needs in one place with its big data cloud service based on Linux.
“Treasure Data monitors and supports its cloud service for customers who collect, store and analyze tens of billions of records a day,” said Eduardo Silva, an open source developer at Treasure Data, who is focused on Fluentd and related open source projects. “Linux is one of our keys to success.”
Here, Silva discusses how Treasure Data uses Linux, the role of Linux and open source in data storage and analytics innovation, and why the company this week joined the Linux Foundation as a new corporate member along with Cirrus Logic and Xilinx.
Linux.com: What does Treasure Data do?
Eduardo Silva: Treasure Data offers analytics infrastructure as a service to help business and technical leaders collect, store and analyze their data when they need it, where they need it. Treasure Data’s managed service makes it easy to collect data from various data sources such as web, mobile, and IoT/sensor data and store them in one place with a unified interface to analyze them.
Treasure Data monitors and supports its cloud service for customers who collect, store and analyze tens of billions of records a day, enabling new applications and granular insights into customers, products and operations. More than 800 users from corporate customers including Pioneer, Yahoo! JAPAN, Pebble, Equifax, Wish.com, MobFox, GREE and other Global Fortune 500 companies access the Treasure Data Service.
How and why do you use Linux?
Silva: From a business point of view, Linux is our primary environment for data collection and big data analysis where we import an average of 500,000 new records per second.
We provide a reliable service and performance is very critical for all markets where our customers are coming from, this includes gaming, IoT, automotive and mobile, among others. Without Linux our position and scenario would be very different. Indeed, Linux is one of our keys to success.
I cannot fail to mention that a large amount of our team comes from the open source world where we have been involved in several projects. The nature of Treasure Data is about contributing and openness. Our more relevant contributions to the ecosystem are: Fluentd, Msgpack and Embulk.
Why did you join the Linux Foundation?
Silva: It was a natural pick and the best decision ever. Besides the fact that the services we provide are built on top of Linux, we share the same motivation to invest in the markets that the Linux Foundation is pushing forward, specifically in embedded (IoT), mobile and automotive. Having a close and strong relationship with the Linux Foundation not only helps our customers, but also the ecosystem and other members with whom we aim to share our experience and open technology.
What interesting or innovative trends in your industry are you witnessing and what role does Linux play in them?
Silva: The growth of data is increasing in unexpected ways, this comes hand-in-hand with the mobile market’s success in recent years. Without a reliable operating system for servers such as Linux, it would be almost impossible to collect, store and analyze data successfully as we are doing now.
How is your company participating in that innovation?
Silva: We participate by creating and sharing open source technology to solve the end-to-end data collection needs and minimize the data loss risk. As mentioned earlier, the projects Fluentd, Msgpack and Embulk are the core of our business and key components of our big data cloud service to process millions of collected events every day. We innovate by building and sharing.
What other future technologies or industries do you think Linux and open source will increasingly become important in and why?
Silva: Definitely the automotive and IoT industries are becoming a big player where Linux and open source technologies are helping to develop and make adoption easier. From our side, we want to make sure that data collection is done right, so we are expanding our business scope to these industries and generating technology to address their needs.
Eduardo Silva is an open source developer at Treasure Data, focusing on Fluentd and related projects. He is the founder of the Monkey HTTP Server Project and Duda I/O, high-performance open-source web framework for Linux. He likes to make software more efficient.
Is your company interested in Linux Foundation membership? See the Corporate Membership page for more information.