Why Containers Alone Can’t Solve Modern Development Challenges

Our industry has made incredible progress over the past decade, thanks in large part to Docker, Docker Compose, and Kubernetes, yet we are still exploring how to develop in today’s diverse environments.

1. Getting Started with Docker

Containerization has sparked a storm in development and operations. In the past, deployment required heavyweight, non‑repeatable engineering work tied to specific technologies—VPS, VM images, static binaries, scripts needing particular interpreters, etc. Many tools such as Capistrano, Puppet, Ansible, deb/rpm packages, cloud‑init, upstart, systemd, and others were used, blurring the line between sysadmin and developer and giving rise to DevOps. As DevOps matured, best practices like the 12‑factor app emerged, though many implementation details remain technology‑specific.

Using Docker for Packaging and Deployment

Docker introduced a simple rule: if an application can be packaged into a container, it can be deployed anywhere. Containers are not new—Google, Solaris Zones, and FreeBSD jails have used them for years—but Docker made portable packaging easy and changed how we deploy applications.

Docker also offers development benefits. Running code in a container that mirrors production provides a consistent environment. It solves version‑dependency issues: different applications requiring MySQL 5.3 and 5.7 can each run in their own containers, started and stopped in seconds.

Using Docker Compose for Development

In late 2013 Docker Compose (originally Fig) entered the scene. It lets developers describe Docker containers and their dependencies in a YAML file, letting Compose manage their lifecycle. It adds features such as log collection, environment variables, and basic container networking for 12‑factor apps, making it an ideal tool for container‑based development.

At first glance Docker Compose seems perfect for local development, but it only supports workflows where everything runs inside Docker. Some scenarios require native processes for performance or OS integration, so developers must stitch together solutions that combine non‑Docker parts with Docker containers.

Because Docker relies on Linux‑kernel features, macOS, Windows, FreeBSD, and other OS users still need a virtualization layer. The complexities of networking, file sync, and VM management we hoped to avoid with containers still exist, and when problems arise we often turn to Google, Stack Overflow, or GitHub for answers.

2. Modern Development: Cloud and Microservices

By 2021, most production‑grade applications depend on cloud infrastructure that cannot run as local Docker containers, introducing new trade‑offs:

Do we stub cloud services? Stubs are cheap and performant but require high engineering effort for anything beyond simple services.

Does each developer get a dedicated instance of every cloud resource? This is costly; serverless options can be cheaper but still require cost considerations.

Do developers share a common development infrastructure? Shared resources lower cost but need extra engineering to avoid conflicts and support multi‑tenancy.

These options are viable in different scenarios, yet using Docker or Docker Compose alone does not solve them, nor does it indicate the best choice. Modern development orchestrators must be cloud‑aware and support varied runtimes. Infrastructure‑as‑code tools come close but focus on production deployment and lack smooth integration with local development environments.

Beyond cloud services, microservices add their own complexity that “just using Docker” cannot address. Large organizations adopting microservices quickly reach a point where every developer would need to run the entire suite of services locally. Tools like Telepresence help connect local containers to remote Kubernetes clusters, but we still lack transparent solutions for service discovery, proxying, and authentication across local and remote environments. Most existing tools are Kubernetes‑centric, raising the learning curve for many developers.

3. What’s Next?

The industry’s progress owes much to Docker, Docker Compose, and Kubernetes, but we still need development tools that can handle local processes, Docker containers, cloud services, and even other teams’ microservices. We don’t have all the answers yet, but we are building “exo” to help developers overcome the complexities of local development.

Original link: https://blog.deref.io/containers-dont-solve-everything