Are Your Kubernetes Workloads Secure? Unsettling Trends in Latest Benchmark
The World Economic Forum says that, despite the economic downturn, we should be prioritizing digital transformation because it enables growth and innovation. Inevitably, digital transformation plans today rely on the scalability and flexibility of the cloud. While launching applications and services in the cloud presents many opportunities, it also comes with new challenges. For example, managing security can be difficult for many organizations as they move production workloads to Kubernetes. This is not surprising—all software has vulnerabilities, and in complex new environments like Kubernetes, it is hard to figure out quickly how to secure all aspects of the open source container orchestration system. Tracking and monitoring workload security over time can pose an additional challenge.
To analyze trends in Kubernetes workload security, Fairwinds gathered data from over 150,000 workloads and hundreds of organizations and assembled the 2023 Kubernetes Benchmark Report. When possible, it also reports changes compared to the benchmark report from the previous year. In a recent CNCF report, 96% of respondents indicated that they were using or evaluating Kubernetes, which is a clear sign that adoption is on the rise. As with any other new technology, however, aligning to best practices can be difficult. That lack of alignment, unfortunately, comes with real-world consequences:
- Increased security risks
- Unmonitored and unmanaged cloud costs
- Reduced reliability of cloud apps and services
Analyzing the data over the past year, Kubernetes workload security is trending in the wrong direction in many of the setting configurations measured in the report.
Running With Insecure Capabilities
We hear the phrase “secure by default” so often that some developers may have come to expect it in newer technologies. In the case of Kubernetes, it is not secure by default. For example, by default, some Linux capabilities are enabled for Kubernetes workloads, even when those workloads do not require those capabilities. The benchmark data shows that organizations are not limiting these capabilities as much as they previously were. In 2021, 42% of organizations turned off these capabilities for most workloads (impacting only 0-10% of workloads). That number dropped to 10% of organizations in the past year.
Allowing Writeable File Systems
You can use the security setting readOnlyRootFilesystem to prevent a container from writing to its filesystem. When this setting is enabled, an attacker cannot tamper with the application or write foreign executables to disk. This security setting is not set to true by default on Kubernetes workloads, so explicitly changing the setting to ensure the most secure configuration possible is vital. In the 2022 benchmark, just 23% of organizations did not override the insecure default setting for 71%-100% of their workloads. Fifty-six percent of organizations failed to make that override in 2022.
Allowing Privilege Escalation
There are some configurations that allow containers to escalate their privileges. Set allowPrivilegeEscalation to false in order to set the no_new_privs flag on the container process. This prevents setuid binaries from changing the effective user ID. It is critical to change that setting when you are using runAsNonRoot. Once again, you must deliberately change this setting to increase security on Kubernetes workloads. The latest report showed that just 10% of organizations locked down most workloads compared to 42% of organizations in 2021.
Enabling Privileged Mode
Use the privileged command to determine whether a container in a pod can enable privileged mode. While a container may not access the host by default, a privileged container has access to the host. If you have this feature enabled, the container has (nearly) the same level of access as processes running on the host. When containers need to use Linux capabilities, such as accessing devices or manipulating the network stack, that ability can prove useful. By default, the privileged flag is off. Unsurprisingly, 87% of organizations were running workloads with the privileged flag off in 2022.
Allowing Run as Root
According to the Benchmark Report, many workloads are allowed this capability. The number of workloads in which running as root is allowed increased in 2022, unfortunately. Analysis showed 44% of organizations were running 71% or more of their workloads allowing root access compared to 22% in 2021.
Using Vulnerable Images
Containers are composed of software, and new common vulnerabilities and exposures (CVEs) are disclosed quite regularly. If you are not regularly scanning container images for vulnerabilities, chances are good that they exist in your container images and deployed containers. According to the report, 40% of organizations had fewer than 10% of workloads impacted by image vulnerabilities in 2021 compared to only 12% of organizations in 2022. One common attack vector for malicious actors is known vulnerabilities, so identifying and remediating these vulnerabilities is important for Kubernetes workload security.
Outdated Helm Charts and Container Images
Outdated Helm charts are a common issue for many organizations. The benchmark report showed that 46% of organizations had 50% or greater workloads impacted by running workloads from outdated Helm charts in 2022. It can be challenging to keep up with updates to Helm charts as release cycles can be unpredictable.
Outdated container images were added to the benchmark in 2022. This data shows that 59% had only 0-10% of workloads impacted, which is good — but 33% had 91-100% of workloads impacted. That leaves a lot of outdated container images. Keeping containers up to date can help minimize the number of containers that contain vulnerabilities, so it is important to know whether your container images are up to date.
Deprecated API Versions
Most organizations have only a few workloads with deprecated API versions. The benchmark data showed that 74% of organizations were up to date with API versions for most of their workloads in 2022 compared to 82% in 2021. Finding and updating or removing deprecated APIs is a critical step in reducing risk during Kubernetes upgrades.
Kubernetes Security Configurations Remain a Challenge
Undoubtedly, the shift to containers and Kubernetes for container orchestration brings significant value to organizations. The challenge today is that many organizations are adopting Kubernetes quickly while others are deploying more applications to Kubernetes than ever before. That is excellent news, but at the same time, many do not yet understand the many configurations available and how to set them appropriately. Use the Kubernetes Benchmark report to understand where Kubernetes is not secure by default, how to make changes so your Kubernetes workloads are more secure, and where your workloads stand in comparison to your peers.
Read the complete Kubernetes Benchmark Report here.