Orchestrating Kubernetes on Bare Metal

Linux container technology permits the subdivision of compute resources in ways similar to virtualization, but without the overhead of managing separate copies of the operating system in each instance/container. To maximize this effect, containers are frequently run on “bare metal” (non-virtualized operating systems). This post explains the approach Cloudify takes to orchestrate the leading container management system (Kubernetes) on bare metal.

Orchestrating Bare Metal with Cloudify

To approach the topic of bare-metal orchestration of Kubernetes, some background in Cloudify orchestration of bare metal is needed. At its core, Cloudify is a distributed task execution engine that is ambivalent about what the tasks are ultimately doing. A blueprint might describe a node that operates a REST API that talks to a well-known cloud (e.g. AWS), a node that copies files or any other programming task you might imagine. In the case of compute resources, blueprints typically describe a unit of code that returns an IP address. An alternative approach is to simply configure the blueprint node with an existing IP address. This is the foundation for the host pool plugin.

The Host Pool Plugin (and service)

The host pool plugin and service manages pools of IP address for Cloudify blueprints. Hosts and their IP addresses can be tagged and filtered so that a blueprint might ask for an IP address to a machine with certain arbitrary characteristics (e.g. “ubuntu” or “large”). For example, a group of physical hosts can be described like so:

This descriptor is consumed by the host pool service and represents the initial configuration. Bear in mind that the configuration isn’t static and can be amended and updated via REST API. In a blueprint, utilizing the host pool plugin, a host can be requested from the pool. The resulting host pool can be treated like any other compute resource for the purposes of orchestration.

Honorable Mention: The Netconf Plugin

While it is likely in a non-virtualized installation of Kubernetes that networking will be set up manually, for completeness it is necessary to include a mention of the Cloudify Netconf Plugin. This plugin provides bare metal orchestration capability in the networking domain (for devices that support Netconf). Of related usefulness is the yttc tool that converts YANG syntax to TOSCA types for inclusion in blueprints.

Kubernetes on Bare Metal

Once the groundwork of understanding and configuring a Host Pool service is complete, the orchestration of Kubernetes is straightforward. An existing Kubernetes blueprint can be adapted to operate on the bare-metal infrastructure. In the case of the referenced blueprint (originally targeting Openstack), the exercise starts with including a reference to the Host Pool plugin and updating the compute nodes in the blueprint. Since we aren’t orchestrating networking in this example, we can remove all relationships and replace Openstack-related config with Host Pool equivalents. The result is something like this (using the sample service filters from above):

BEFORE

AFTER

The remainder of the effort is largely removing all networking configuration. Autoscaling and healing will function as expected, assuming the host pool has sufficient hosts defined. Note also that the host pool configuration isn’t fixed and additional machines can be made available by updating the host pool service via its REST API.

Conclusion

This post explained how Kubernetes can be managed by Cloudify on bare metal by a simple modification of already available blueprints. This is a good example of the power of the compute abstraction in TOSCA modeling, made possible by the fact that both the pre-conversion Openstack Server type and the post-conversion Host Pool Host (or in this case the derived LinuxHost type are both derived ultimately from the cloudify.nodes.Compute type, which makes them easily interchangeable in blueprints. It also demonstrates how, and why, Cloudify orchestration is not limited to clouds or virtualization.

About the Author / DeWayne Filppi

DeWayne Filppi is a solution architect in the CTO office at GigaSpaces, working on Cloudify.  He is a software technologist with broad and deep industry experience, ranging from product evangelism, pre-sales engineering, post-sales consulting, through product design, development, architecture, and management. He is focused on high performance server platforms and cloud computing.  Lately he has been tinkering around with fun stuff around open source tooling.  Read his posts on the Cloudify Blog, or follow him on Twitter & Github.