Why You Shouldn’t Use a Jump Server to Protect Kubernetes’ API

Kubernetes has become the de facto standard in running and managing workloads. A Kubernetes cluster comprises multiple resources (such as machines, CPUs, memory, storage, etc.) and software components with multiple purposes. The user of a Kubernetes cluster communicates with a cluster using the Kubernetes API, which is implemented by a Kubernetes API server.

The Kubernetes API server supports rich authentication and authorization mechanisms and enforces high-level security with mutual TLS or OpenID Connect. It also enables role-based access control (RBAC) with fine-grained access control.

This means that the Kubernetes API is critical to Kubernetes security and plays a major role in most threat models. If an attacker gets full access to the Kubernetes API, it’s game over, man.

For this reason, organizations tend to put extra effort into securing access to the Kubernetes API. What tools should we use to protect this API? When considering that question, old habits die hard. A widespread approach is to apply network segmentation principles to this problem, creating rules which disable access to the API server from outside the virtual private cloud (VPC)—and, therefore, from the public internet.

That’s all well and good, but sometimes there’s still a need to access the Kubernetes API server from outside the VPC. In such a case, the operator will create a “jump” or “bastion” server in the VPC, which can be accessed via SSH from the public internet. SSH is secure as long as the client’s private key is kept confidential. When the user logs in to the jump server, a shell, which is set up to work with the cluster directly, is waiting–most likely with cluster administration privileges.

Is this secure? Yes, it is.

But is it more secure than just giving the API server itself access to the public internet? This may not be intuitive, but no; using a jump server is no more secure than exposing the API server.

Let’s look at this from the attacker’s point of view.

If the Kubernetes API server is publicly reachable, the attacker must first beat the server’s authentication mechanisms. These will either be TLS client certificate authentication or some kind of token authentication. Both are based on industry-standard quality cryptography (if you don’t trust these protocols, I suggest not using your credit card online!).

If an attacker could overcome this barrier, they most likely could get their hands on leaked or stolen credentials. Unless those credentials belonged to a cluster administrator, the attacker is probably still limited by Kubernetes role-based access control policies, despite their initial success.

If, however, the Kubernetes API is protected by a jump server, the attacker needs to beat SSH authentication. Its cryptography is based on the same industry standards as the Kubernetes API authentication above. However, the jump/bastion machine already has a Kubernetes environment working in the SSH shell, most likely with full cluster administrator privileges. An attacker that overcomes this barrier has all the privileges they need to access the cluster.

There are obviously ways that the jump server setup can be made more secure, requiring additional security steps after SSH login and limiting the privileges of the user who logged into the SSH shell could work, hypothetically. However, all these steps are not standard and are hard to enforce and maintain in practice.

Returning to the original question of whether the jump server setup is more secure, my answer, again, is “No.” Using a jump server is based on an old instinct–an instinct that’s not completely invalid, but which, in most cases, doesn’t enhance security. Worse, it can sidestep practical security measures such as RBAC.

It is important to get authentication and authorization right in Kubernetes, but that’s true for SSH too. So why deal with two security configurations if one is just as secure?

Ben Hirschberg

Ben Hirschberg is CTO of ARMO, creators of Kubescape, the open source Kubernetes security platform.

Ben Hirschberg has 2 posts and counting. See all posts by Ben Hirschberg