Portworx this week launched an open source STorage Orchestrator Runtime for Kubernetes (STORK) project, which promises to make it easier to manage the underlying storage systems serving containerized applications.
Company CTO Gao Rau says STORK provides a mechanism to inform Kubernetes scheduler software about the state of the physical storage systems attached to a Kubernetes cluster. That capability is needed because when it comes to stateful applications IT operations teams continually deal with issues such as failed servers and drives, as well as activities such as garbage collection to free up storage space, says Rau.
STORK is designed to communicate with storage drivers via a plugin interface, which Rau says means it can be extended to work with any storage driver for Kubernetes. Critical storage management capabilities addressed by STORK include enabling hyperconvergence for containerized applications, monitoring of physical storage and support for volume snapshots to simplify backup and recovery.
In general, Rau says many organizations underestimate the complexities associated with managing container storage. There can be hundreds of containers per host, all trying to access the same physical storage. That creates a lot of I/O contention in environments that employ bare-metal servers. In many cases, that issue results in IT organizations opting to deploy containers on top of virtual machines. But that strategy limits the number of containers that can be deployed on a host in addition to adversely impacting performance. Rau notes that software such as Cassandra, Kafka, ElasticSearch perform best when each instance runs as close to a data source as possible.
Rau says advances such as Rook storage software and the Container Storage Interface (CSI) represent major advances in terms of making it practical to deploy stateful applications based on containers in a production environment. But there is still a need to inform the scheduler what storage resources should optimally be invoked at any given time, says Rau. Once that issue is resolved, there should be many more instances where containers are deployed on bare-metal servers.
IT organizations that have deployed containerized applications have been relying on labels and persistent volume claim (PVC) capabilities within a cluster to get around these issues. But Rau says PVC requires reliance on labels being employed within the context of affinity rules for an application that are complex to implement and manage. For example, he notes IT organizations can’t dynamically add labels because they don’t know the PVC names when creating a stateful set of data. When there are thousands of containers running, DevOps teams find managing labels at scale impractical, he says.
Rau says most of the decisions relating to storage are being made by DevOps teams trying to optimize storage for either a greenfield application or legacy application that is being lifted into a Docker container. Traditional storage administrators today are not major participants in that decisions process, says Rau. But as storage increasing becomes part of the mainstream container conversation in IT circles, it’s only a matter of time before more storage administrators get pulled into that discussion.