The Technical Oversight Committee (TOC) within the Cloud Native Computing Foundation (CNCF) this week has decided to accept Vitess as a new project.
Vitess is a database orchestration system developed at YouTube that automatically shards instances of open source MySQL databases as they horizontally scale.
Sugu Sougoumarane, CTO at PlanetScale Data and co-creator of Vitess during his tenure at YouTube, says Vitess enables organizations to shard databases in a way that is transparent to applications. They can even split and merge shards as needs grow by employing an atomic cutover step that takes only a few seconds. That’s critical because each time a database is broken up into shards developers would otherwise have to manually adjust the applications accessing that database. Given the number of times a database needs to be broken up into shards as an application scales, the processes associated with rewriting applications to deal with each new shard of a database becomes unwieldy to manage and is highly prone to errors, Sougoumarane says, adding all that manual effort also increases the risk of introducing additional security vulnerabilities.
Active contributors to Vitess besides Google include Booking.com, GitHub, HubSpot, Slack and Square, and all are also active contributors to the project. Companies that make use of Vitess today also include BetterCloud, Flipkart, Quiz of Kings, Slack, Square Cash and kStitch Labs.
Over the last two years contributors to Vitess have been optimizing the database orchestration system for the Kubernetes container orchestration platform that is overseen by CNCF. Both Kubernetes and Vitess trace their lineage back to Google, but Vitess can be deployed on platforms other than Kubernetes. Vitess has been under development since 2010. The earliest version was a connection proxy, but over time it has evolved considerably.
Many IT organizations have embraced so-called NoSQL databases as an alternative to traditional relational databases in part because managing multiple shards of databases is too difficult. Alternatives to relational databases make it easier to manage large amounts of scale. But the trade-off is often performance. The CNCF decision to take Vitess on as a project should make it more attractive to deploy instances of MySQL on top of Kubernetes at scale.
Making it easierer to deploy and manage relational databases on Kubernetes is critical for deploying mission-critical stateful applications. Most container applications deployed today are stateless. But as advances in container storage technologies continue to be made, it’s becoming easier to deploy stateful applications based on databases accessing various forms of persistent storage. Vitess makes it possible to arguably manage a relational database at scale running on Kubernetes than it otherwise might be on another platform.
How many of those relational databases get deployed on bare metal or virtual machines running in the cloud or on-premises remains anybody’s guess. But what is certain is that MySQL will not be the only type of relational database being deployed on Kubernetes. At the moment, however, given the widespread usage of MySQL, coupled with an ability to transparently shard databases instances as applications scale, it’s likely MySQL will be one of the most popular databases to be deployed on Kubernetes for some time to come.