Prometheus Monitoring Ecosystem Begins to Mature
The ecosystem surrounding the open source Prometheus monitoring systems for Kubernetes clusters is starting to mature.
Cortex, a scalable, highly available, multi-tenant storage platform for Prometheus, and Thanos, a metric system that provides a way to centralize and scale instances of Prometheus, have both moved up to become incubation-level projects within the Cloud Native Computing Foundation (CNCF). The next step would be for the projects to move up to the same graduated level as Prometheus and Kubernetes, assuming they are able to garner enough support across the CNCF community.
Frederic Branczyk, a system engineer for Red Hat who is a maintainer of Thanos, notes each instance of a Kubernetes cluster requires an instance of Prometheus to monitor it. Thanos provides a mechanism to aggregate all the data across fleets of Kubernetes clusters to provide IT teams with a global view of a Kubernetes environment, he says.
It is used in production by Alibaba Cloud, Banzai Cloud, HelloFresh, Monzo and Red Hat, among others, with seven maintainers from four different companies: AdForm, Grafana Labs, Red Hat and Utility Warehouse.
As a complementary project, Cortex provides a way to reduce the cost of storing all that data for organizations that want to compare metrics over extended periods of time, adds Branczyk. The project now has eight maintainers from four different companies—Grafana Labs, Microsoft, Splunk and Weaveworks—and is used in production by several organizations at scale, including EA, Gojek and Rewe Digital.
Use of Prometheus appears to be keeping pace with adoption of Kubernetes clusters, especially in application development environments. Developers who adopt Kubernetes tend to also prefer to adopt complementary tools that don’t require them to obtain approval from a purchasing manager. As container applications move into production environments, it’s less clear how many organizations will continue to employ Prometheus—most IT organizations already have a plethora of monitoring tools at their disposal, many of which are now being extended to consume data generated by Prometheus agent software. That approach allows them to employ the same monitoring tools for both legacy IT environments and emerging cloud-native applications.
However, there are greenfield Kubernetes deployments that will also wind up relying solely on Prometheus to monitor Kubernetes in a production environment. It is likely only a matter of time before more of those environments require the capabilities enabled by Thanos and Cortex. Most of those greenfield applications are based on microservices that require some level of observability to enable IT teams to discover, navigate, manage and eventually secure all the dependencies that exist across potentially hundreds of microservices.
Observability is, of course, a core tenet of any best DevOps practices. In fact, Kubernetes and microservices require IT teams to embrace DevOps. Prometheus isn’t the only way to achieve the observability required to drive DevOps. However, there’s no doubt Prometheus is helping to accelerate the adoption of best DevOps practices, especially as the tools needed to store and make sense of all that data continue to mature.
