May 26, 2017

Docker, the DevOps darling is growing up, and DockerCon 2016 was proof of that.

It seems like just yesterday I did a pull of my first image from the hub, LAMP or something, and was off and running on my local machine. That was January 2014 when the hub downloads were sub-10 million.

But now the downloads are more than 4 billion and Docker has 5,000 paying customers. Docker is growing up and fast. And part of that growth is demonstrating that the tool is more than a developer sandbox. With the announcement this year at DockerCon of version 1.12 with Swarm baked in, it is clear that Docker is turning its attention to more mature deployments that extend far beyond the developer box.

Not only did Docker bake in Swarm, it also added a new service deployment API and in-memory caching. That makes Docker a more viable replacement for Kubernetes and Mesosphere, and it could mitigate the risk of those tools dominating the container driven data center. And that ultimately is what I felt like Docker and DockerCon were about this year—nothing groundbreaking, but the company did establish its position as an enterprise-level player.

A couple of the themes Docker built around this year were “democratization of containers” and “batteries included,” as in, “We put Swarm in the box in case you need orchestration.” But the former I think is hard to justify, considering that the announcements were the opposite of “democratization.” They are much closer to lock-in than anything Docker has announced previously. And this is a consideration anyone will have to make if they adopt Swarm fully.

Docker Partners

That brings me to partners.

With some of the new functionality, namely orchestration, my first thought was, What about the Docker partners? A large portion of the 450 or so third-party tools are built around orchestration and maintenance of containers by providers such as Rancher, Aqua and Codefresh. Even if the new functionality is part of a broader platform strategy, to leverage new features of Swarm the partners would have to rearchitect many aspects of their product—which they cannot do overnight, or at all if they are small. Ben Golub, CEO of Docker, did reassure us DockerCon that “we are religious about backward compatibility”—so at least things won’t break.

In some areas, however, any improvements to the platform will only help the ecosystem. Take, for example, the Docker Volume API for building volume plugins. “Any new functionality they add only helps us build new functionality,” said Rob Whiteley, vice president of Marketing for Hevdig. Its product uses the volume plugin to interface with their storage logic.

Docker will continue to build what the customer asks for. And right now the customer is asking for all the the tooling that makes containers run beyond the developer box. This clearly is going to mean functionality that the its partners already have built ahead of the curve. They will continue to support the partner channel, but not at the expense of slowing development.

I heard from several vendors during DockerCon that two years ago as a Docker partner they got no attention, but today they have an individual they can actually talk to. The evolving Docker means we can expect to see a channel that is confined to niche tooling. There will be limited growth, but partner innovations still will be necessary. For example, the channel will help to provide richer visualizations—an area in which Docker technology, built by developers for developers, has been weak—through companies such as Weaveworks, DCHQ and Calm.io.

Docker in Production

The new functionality also goes a long way toward supporting production deployments. According to Golub at DockerCon, a recent survey has shown that 60 percent of Docker users are running the platform in production. He later clarified that it ranges from 30 percent to 60 percent. But even with an obvious selection bias, of all the container surveys put out there, this is a very high number—and one I have yet to validate in the field, which has seen rates closer to around 5 percent to 10 percent. DataDogs’s brand new analysis of Docker’s user base clearly shows adoption is healthy and up and to the right, which is what matters the most.

I suspect the discrepancy between the various statistics on Docker adoption is in the definition of what production is. Even your internal weather app is considered production. My interest is in public-facing applications that companies live and die by. Currently, adoption seems to be confined mostly to the developer. Even when Docker talks about growth it usually references the number of hub downloads first. But the public registry and the new Docker Store, also announced at DockerCon, are not likely to be used in a mature delivery chain. Image downloads may be a big number, but it is also just an indication that even Docker knows developers in web scale companies are still their biggest fans.

Docker for Back Ends

Alongside and a part of production is the other commonly referenced limitation for Docker—supporting those portions of the application that are heavy and need persistence. Docker 1.12 introduces some small enhancements to the volume driver, such as identifying local or global accessibility, and it’s these incremental enhancements that we can expect more of. But the partners certainly care about storage, and man, there are a lot of them: pure-play storage systems NetApp and EMC, as well as HPE, Hevdig, Portoworks, Platform9, Virtuozzo, Robin Systems and more. Even SUSE has put a lot of thought into it with SUSE Enterprise Storage.

The EMC REX-Ray team in particular is doing some very interesting things by supporting a robust open-source project that does automatic linking of persistent volumes to provisioned containers. What is most impressive is that its tool is being used in public clouds as well. That demonstrates EMC’s support for moving the needle forward.

Many believe persistence does not really fit in the container world view, or wonder whether there is even true interest in persistent storage. They believe that back ends and containers are to remain forever separate, since they are built by different teams and have different life cycles.

Personally, I refuse to accept that logic, and Josh Bernstein, vice president of EMC {code} offered proof. “Look at the top downloaded images in the hub, and you will notice that seven of the first 12 are databases, and most of the time databases need persistent storage,” he said. While those databases may not be Oracle or MS SQL, they do prove that developers want their back ends containerized as well.

They want the same of their monolithic line of business applications. “Any application written in the last 15 years can be dockerized,” said Golub at DockerCon. “Docker is sometimes even adopted as a migration path for line-of-business applications. Companies will put their heavy application in a container. It is big and stateful, but they still get the benefits of containers and can slowly start breaking down the applications into what we believe are modern architectures. We want incremental revolutions.”

DockerCon Shows, the Money is Flowing

Docker and containers are beyond the hype stage, and clearly making good headway into being part of enterprise development shops. “About one-fourth of our attendees are IT Ops,” Golub notes. And there sure is a lot of money at play. All the big companies are doing their part: Consider NetApp’s acquisition of SolidFire, HPE’s investment in Mesos and what seems to be some serious love in return from Docker, given HPE’s constant promotion of built-in Docker on each HPE server. Don’t forget Microsoft’s Windows containers, EMC’s REX-ray and even outsiders like Samsung, which recently acquired Joyent.

I appreciate Docker’s openness, and its ability to listen to the market, which I can safely say is the best I’ve seen in a 200+ employee organization. I still think the fight to bring containers to enterprises and production is ongoing. But as DockerCon showed, it’s no longer about awareness—it’s about leveraging the new functionality in 1.12 and Swarm, such as discovery and self-healing, and strong case studies such as ADP and the GSA—and then, my friends, VMs might truly become the cassette tape of the modern development world.

Chris Riley

Chris Riley (@HoardingInfo) is a technologist and DevOps analyst for Fixate IO. Helping organizations make the transition from traditional development practices to a modern set of culture, tooling, and processes that increase the release frequency and quality of software. He is an O'Reilly author, speaker, and subject matter expert in the area of DevOps Strategy, Machine Learning, and Information Management.