Ways to Extend the Core Kubernetes API

Kubernetes has become a powerful tool for container orchestration. We’ve already covered the benefits of utilizing the Kubernetes REST API to improve the Kubernetes experience, but did you know you can further extend the API?

In this article, we detail how to extend the Kubernetes API with the aggregation layer, API server extensions and custom resources. We’ll see what sort of advanced functionality we can add to the Kubernetes API for nuanced customization.

What is the Aggregation Layer?

We know that Kubernetes is a powerful microservices orchestrator, and using the API can add a programmatic touch. The aggregation layer takes this a step further, allowing Kubernetes to be extended with additional APIs, beyond what is offered by the core Kubernetes APIs.

The aggregation layer enables you to create Kubernetes-style APIs within a cluster. These APIs can be user-generated, such as through Kubebuilder, the SDK for building Kubernetes APIs using custom resource definitions (CRDs). Alternatively, these extensions could come as third-party kits such as Service Catalog, a service that enables you to provision cloud services from within native Kubernetes tooling.

You must first register an API by adding an APIService object to define URL paths. The aggregation layer then works by proxying requests sent to said path. A path may look something like:

““

/apis/myextension.mycompany.io/v1/

““

An extension API server running in a pod in a cluster will typically implement the APIService.

Configuring the Aggregation Layer

To begin, you will need a Kubernetes cluster, along with the kubectl CLI. If you don’t already have a cluster, you can create one using Minikub.

First, we configure the aggregation layer to the API server. As stated in the documentation, these apiserver flags will enable the aggregation layer:

““

–requestheader-client-ca-file=<path to aggregator CA cert>

–requestheader-allowed-names=front-proxy-client

–requestheader-extra-headers-prefix=X-Remote-Extra-

–requestheader-group-headers=X-Remote-Group

–requestheader-username-headers=X-Remote-User

–proxy-client-cert-file=<path to aggregator proxy cert>

–proxy-client-key-file=<path to aggregator proxy key>

““

It’s quite possible that your environment already has them installed.

Set Up API Extension Server

You must set up an extension API server to work with the aggregation layer. This sample repo provides a demonstration of how what it will look like.

Once it is set up and keys are in place, initiating a server will look something like:

““

etcd &

sample-apiserver –secure-port 8443 –etcd-servers http://127.0.0.1:2379 –v=7 \

–client-ca-file ca.crt \

–kubeconfig ~/.kube/config \

–authentication-kubeconfig ~/.kube/config \

–authorization-kubeconfig ~/.kube/config

““

Further tips on setting up the Extension API Server can be found here. Alternatively, the apiserver-builder library can be utilized.

About Custom Resources

In Kubernetes, a typical resource is a collection of API objects. Therefore, custom resources are customized installations that aren’t necessarily within every cluster.

Using dynamic registration, they can be triggered on or off, and they can be affected independently of the cluster by administrators. Once custom resources are installed, they can be acted upon with kubectl, similar to how you interact with pods.

Understanding Custom Controllers

Custom resources are simply structured data that can be fetched, but controllers enable you to declare the state of a resource; hence making it a declarative API. Custom controllers can be deployed on any cluster with any resource type.

Regarding custom resources and controllers, the documentation reminds us to consider aggregation with the entire cluster if the API is declarative, and consider building a standalone API if it is not declarative.

API Aggregation Offers Advanced Flexibility Over CRDs

Though CRDs are simple to create with less programming, API aggregation offers more flexibility, such as customizing how data is stored, or how API versioning is handled.

As defined in this comparison chart, aggregated APIs offer features that CRDs do not, allowing endpoints to support `PATCH` HTTP verb, enabling protocol buffers and going beyond CRUD, with `logs` or `exec`.

Additional Features

The advantage of creating custom extensions with the Kubernetes APIs is that many features come out of the box. Some of these include API “watches” for efficient change notifications, enhanced discovery methods to display field operations, HTTPS and built-in authentication and authorization. There is also client generation, a UI and CLI to display extension resources and many other features. Essentially, the benefits of the Kubernetes API platform carry over to custom resources quite well.

Final Thoughts

This is meant to be an introduction into the power of extending the Kubernetes API beyond its core components, outlining high-level steps involved in doing so. For more specific instructions, peruse the sample-apiserver repo to get going.

Though more complex to implement, API aggregation, as opposed to CRDs, equates to the greatest flexibility. Extending the Kurbentes API with custom resources can add both flexibility and power to your Kubernetes environment. So give it a shot, and let us know how it goes below!

Bill Doerrfeld

Bill Doerrfeld

Bill Doerrfeld is a tech journalist and analyst based in Seattle. His beat is cloud technologies, specifically the web API economy. He began researching APIs as an Associate Editor at ProgrammableWeb, and since 2015 has been the Editor at Nordic APIs, a high impact blog on API strategy for providers. He loves discovering new trends, researching new technology, and writing on topics like DevOps, REST design, GraphQL, SaaS marketing, IoT, AI, and more. He also gets out into the world to speak occasionally.

Bill Doerrfeld has 6 posts and counting. See all posts by Bill Doerrfeld