What will solve the persistent data storage challenge for containers? CoreOS says the answer is Torus, a new open source distributed storage platform it announced June 1.
According to CoreOS, which announced the technology in a blog post, the storage system aims to solve three main challenges:
- Existing storage solutions were not designed for large clusters and consequently don’t work well with them.
- Some existing distributed storage solutions require expensive proprietary hardware.
- Persistent storage for container clusters is different from persistent storage for physical or virtual servers. With containers, you need storage that is “uniformly available network-wide” and remains consistent even as data processing shifts between different containers, according to CoreOS.
Torus addresses these challenges, CoreOS says, by providing a flexible storage format that can take the form of block devices or large object storage. And although the storage system is distributed, it provides a simple access interface that looks like a traditional file, “allowing for storage manipulation through well-understood basic file operations,” according to CoreOS.
It’s clear from the Torus announcement that the system currently remains basic. It offers most of the essential storage features, including consistent hashing, replication, garbage collection and pool rebalancing, and it exposes storage via a Network Block Device. But CoreOS says that it envisions expanding these features so that other types of storage systems can be built on top of Torus. The company also plans to add encryption and error correction “in the near future.”
Torus and the Future of Container Storage
Given that Torus still needs some work to be production-ready, it’s not going to revolutionize data storage for containers right away. But it’s a significant move for CoreOS, which can now claim to have engineered a complete distributed storage solution that doesn’t depend on other storage technologies.
That differentiates Torus from other container storage answers such as Docker Data Volumes. They make container data storage persistent, which is one big part of the puzzle, but they still rely on underlying storage systems, such as the host’s file system, to store the actual data. Docker Data Volumes and data volume containers might be better fits for some situations. And so far, there’s sign that they won’t be compatible with Torus, since the two tools work at different levels. But they don’t provide the comprehensive answer that CoreOS wants to deliver with Torus.
For that reason, Torus is likely to appeal to developers searching for something more elegant than other container data storage techniques, which can feel like an attempt to tack storage awkwardly onto containers rather than integrating the storage system seamlessly into the rest of the infrastructure.