CNCF Adopts Database Project
The Cloud Native Computing Foundation (CNCF) is now hosting an open source distributed transactional key-value database optimized for stateful applications running in a container.
Based on concepts put forward by Google in the form of a Spanner database service it provides as a cloud service, the TiKV database is designed from the ground up to automatically scale out as more compute and storage resources are required by containerized applications.
Kevin Xu, general manager of PingCAP, which contributed TiKV to the CNCF, says traditional databases are designed on the assumption that compute resources will scale up rather than out across a cloud computing environment. To make it simpler to move existing applications to TiKV, the database also supports the same interfaces as the widely employed open source MySQL database that is now owned by Oracle via TiDB, a stateless MySQL-compatible SQL layer also built by PingCAP. Unlike MySQL, however, TiKV eliminates scale concerns that have long troubled IT organizations that have tried to employ MySQL to run traditional web applications at scale, says Xu.
That capability is critical because most organizations are looking with modern stateful applications to meld transaction processing and analytics in a way that requires compute and storage resources to be able to scale dynamically on a truly elastic basis, says Xu. Accomplishing that goal using legacy relational databases requires complex “sharding” of databases that become increasingly difficult to manage over time, he says.
Built using the Rust programming language, the TiKV database project employs the Raft consensus algorithm to maintain state across a distributed database spanning multiple clusters. Contributors to the TiKV project, which launched in 2016, include Samsung, Mobike, Toutiao.com, Ele.me, Tencent Cloud and UCloud. Xu says more than 200 companies are already using TiKV in production environments, including Bank of Beijing, Ele.me, Hulu, Lenovo and Mobike.
It’s too early to say whether an open source database capable of supporting high rates of transactions using a scale-out architecture represents an existential threat to legacy relational databases. But it is clear that as more stateful applications are deployed using containers, the underlying database architecture will need to evolve. Thousands of microservices based on containers will inevitably create I/O bottlenecks as well as a need to dynamically scale out compute resources. TiKV provides IT organizations an opportunity to accomplish that goal consistently across on-premises and cloud services environments.
In the meantime, the rate at which stateful applications are being containerized varies by organization. Many organizations are simply hosting databases in containers as a means of lifting and shifting them into the cloud without having to refactor their applications. But in time databases—much like the rest of the IT environment—will become more “cloud native.” In fact, rather than there being a single database format, it’s probable microservices will be accessing a much wider variety of data stores. To facilitate that transition, Xu says PingCAP plans to make available a database-as-a-service offering based on TiKV and TiDB. Of course, now that TiKV has become a CNCF project, chances are high PingCAP will have plenty of competition.
