Kubernetes has become a powerful tool for container orchestration. We’ve already covered the benefits of utilizing the Kubernetes REST API to improve the Kubernetes experience, but did you know you can further extend the API?
In this article, we detail how to extend the Kubernetes API with the aggregation layer, API server extensions and custom resources. We’ll see what sort of advanced functionality we can add to the Kubernetes API for nuanced customization.
What is the Aggregation Layer?
We know that Kubernetes is a powerful microservices orchestrator, and using the API can add a programmatic touch. The aggregation layer takes this a step further, allowing Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs.
The aggregation layer enables you to create Kubernetes-style APIs within a cluster. These APIs can be user-generated, such as through Kubebuilder, the SDK for building Kubernetes APIs using custom resource definitions (CRDs). Alternatively, these extensions could come as third-party kits such as Service Catalog, a service that enables you to provision cloud services from within native Kubernetes tooling.
You must first register an API by adding an APIService object to define URL paths. The aggregation layer then works by proxying requests sent to said path. A path may look something like:
An extension API server running in a pod in a cluster will typically implement the APIService.
Configuring the Aggregation Layer
To begin, you will need a Kubernetes cluster, along with the kubectl CLI. If you don’t already have a cluster, you can create one using Minikub.
First, we configure the aggregation layer to the API server. As stated in the documentation, these apiserver flags will enable the aggregation layer:
–requestheader-client-ca-file=<path to aggregator CA cert>
–proxy-client-cert-file=<path to aggregator proxy cert>
–proxy-client-key-file=<path to aggregator proxy key>
It’s quite possible that your environment already has them installed.
Set Up API Extension Server
Once it is set up and keys are in place, initiating a server will look something like:
sample-apiserver –secure-port 8443 –etcd-servers http://127.0.0.1:2379 –v=7 \
–client-ca-file ca.crt \
–kubeconfig ~/.kube/config \
–authentication-kubeconfig ~/.kube/config \
About Custom Resources
In Kubernetes, a typical resource is a collection of API objects. Therefore, custom resources are customized installations that aren’t necessarily within every cluster.
Using dynamic registration, they can be triggered on or off, and they can be affected independently of the cluster by administrators. Once custom resources are installed, they can be acted upon with kubectl, similar to how you interact with pods.
Understanding Custom Controllers
Custom resources are simply structured data that can be fetched, but controllers enable you to declare the state of a resource; hence making it a declarative API. Custom controllers can be deployed on any cluster with any resource type.
Regarding custom resources and controllers, the documentation reminds us to consider aggregation with the entire cluster if the API is declarative, and consider building a standalone API if it is not declarative.
API Aggregation Offers Advanced Flexibility Over CRDs
As defined in this comparison chart, aggregated APIs offer features that CRDs do not, allowing endpoints to support `PATCH` HTTP verb, enabling protocol buffers and going beyond CRUD, with `logs` or `exec`.
The advantage of creating custom extensions with the Kubernetes APIs is that many features come out of the box. Some of these include API “watches” for efficient change notifications, enhanced discovery methods to display field operations, HTTPS and built-in authentication and authorization. There is also client generation, a UI and CLI to display extension resources and many other features. Essentially, the benefits of the Kubernetes API platform carry over to custom resources quite well.
This is meant to be an introduction into the power of extending the Kubernetes API beyond its core components, outlining high-level steps involved in doing so. For more specific instructions, peruse the sample-apiserver repo to get going.
Though more complex to implement, API aggregation, as opposed to CRDs, equates to the greatest flexibility. Extending the Kurbentes API with custom resources can add both flexibility and power to your Kubernetes environment. So give it a shot, and let us know how it goes below!