Building Security into the 3 Phases of Container Deployment

Container adoption has skyrocketed in recent years, and the importance of securing the applications DevOps teams create and deploy using containers has grown along with it. The protection security teams provide must be comprehensive across the entire container life cycle, and built into the DevOps pipeline in a way that’s seamless and unobtrusive. Accomplishing this requires an understanding of Docker container technology, and the adoption of processes and tools tailored for these environments.

Container Security Challenges

Containers create particular security and compliance issues, with many stemming from the qualities that developers find attractive, such as packaging an application and its dependencies without a guest OS. These challenges include the use of unvalidated software from public repositories, which often has unpatched vulnerabilities, and the deployment of containers with weak configurations.

In addition, containers communicate directly with each other via exposed network ports in a way that bypasses host controls, and they’re difficult to track because they’re so ephemeral.

Container Security Goals

Security teams should pursue four main goals for container security, starting with obtaining visibility across their container environment by discovering and tracking all deployments—on-premises and in clouds.

Next, they need to both manage vulnerabilities within containers, and prevent and detect intrusions. A third goal should be to take advantage of REST APIs to integrate DevOps products with container security tools.

Finally, the container security program should rethink incident response (IR), especially since vulnerable containers aren’t “patched” but rather replaced with new ones from an updated image.

Teams should update the IR program to collect information related to the new operating environment for containers, such as the orchestration platform, and plan for the “rip and replace” model as an IR element.

Protecting the Container Pipeline

All three phases of container deployment—build, ship and runtime—have to be secured, and each has specific requirements. Let’s look at each one in detail.

Build

In this first phase, the primary security goal is to block vulnerable images from entering your organization’s repositories. This is especially important for images that aren’t created from scratch but are pulled whole or in part from a public repository—a very common practice among container developers.

To keep these unsafe images out of your Docker repositories, it’s important to leverage REST APIs or native plug-ins so that security checks can automatically run within your DevOps team’s preferred CI/CD tools. You should have the flexibility to determine parameters for blocking images, such as the presence of vulnerabilities with a severity rating of 3, 4 or 5; a lack of adherence to compliance standards; and usage of secrets openly in the images.

These integrations also will allow the security team to grant developers access to the security tools. With this access, developers proactively can perform certain security functions right from their CI/CD tool without having to rely on—and wait for—the security team. For example, developers will get access to scan-result data and be able to take remediation actions right from their CI/CD tool’s interface.

Ship

In this phase, it’s key to make sure images that are already in your repositories are checked for vulnerabilities. Registries and repositories should be inventoried, and as images are added they should be scanned for vulnerabilities. You should also schedule automated daily scans to check for new vulnerabilities and to check the new images being added to the repositories daily.

You also should check that images in your organization come from sources that are trusted and reputable and keep their images current and scrubbed of disclosed vulnerabilities. Use notary services, such as Docker’s Notary service, or other similar services to sign images and ensure only trusted images are being used in your environment.

Runtime

With the container images now on your registry and available for production use, it’s critical to have visibility and continuous monitoring of runtime environments as well as the ability to prevent and respond to breaches.

Security teams must detect rogue or vulnerable containers, identify where they are and assess their potential impact based on how widespread they are in your environment.

A key here is to flag containers already running on the system that are breaking off from the “immutable” behavior of their parent image, which could indicate a breach.

Containers follow the image, so it’s key to identify the behavior of the containerized application and detect any suspicious deviations, such as unexpected system calls, processes and communications.

Security teams must determine where these images are cached in the runtime environment, and identify both active and dormant ones.

After identifying rogue containers, your security tool should allow you to enforce counter measures of blocking or quarantining, and allow you to drill down into the details of the anomalies, so you can understand what the issues are. This shouldn’t affect your operations because you can spin up a new container from the parent image to replace the rogue one.

A better option would be to automatically validate the image against security policies using your tool and block unapproved images from being spun up as containers. Here you can leverage orchestrators such as Kubernetes to prevent rogue containers from entering the environment via admission controllers.

In addition, follow security practices prescribed for the orchestration environments to have a secure, “least access” model operating environments. If you have access to the underlying host, harden it for vulnerabilities and compliance requirements.

Summary

We hope this article has helped you better understand the particular security challenges of containers, and how to address them throughout their life cycle: blocking vulnerable images from entering repositories during the build phase; securing images pushed to your registries in the ship phase; and scanning in production during the runtime phase to identify and manage compromised containers.