Architects' Roundtable: Container Practice and Orchestration Experiences

The architects' group organized a session where each episode focuses on a hot technology topic; the first episode centered on container practice, highlighting Docker’s disruptive impact on CI/CD, DevOps, and micro‑services, while acknowledging its maturity challenges in production.

Participants included Hujiang architect Huang Kai, Didi architect Tian Zhiwei, Mogujie architects Zhang Zhenhua and Xiang Jing, Qiniu technical director Yuan Xiaopei, and Shanbay technical director Ding Yan.

Hujiang’s Huang Kai described moving from a simple architecture to Docker on bare metal, using Mesos and Marathon for orchestration, and running FFmpeg‑based transcoding services inside containers, emphasizing the efficiency gains over VM‑based Docker.

Didi’s Tian Zhiwei explained their early Docker adoption for business systems and middleware, the creation of a pre‑release Docker environment, and a separate Docker+K8s based offline CI/CD platform, focusing on namespace‑based configuration isolation and the trade‑offs between Docker‑only and Kubernetes‑based solutions.

Mogujie’s Xiang Jing outlined a hybrid cloud PaaS plan combining Docker, VMs, and public‑cloud resources, discussing resource abstraction, configuration isolation via environment variables, and security considerations of mutable host environments.

Mogujie’s Zhang Zhenhua recounted early Docker usage (v1.3.2) with OpenStack, the pitfalls of treating Docker as a VM, and the subsequent shift to Kubernetes for better resource utilization, CI/CD, and isolation.

Qiniu’s Yuan Xiaopei highlighted massive data‑processing workloads (hundreds of billions of daily requests), using containers for rapid scaling, kernel‑level isolation, and a custom Mesos‑based scheduler, while employing Consul for cross‑data‑center service discovery and preferring Bridge networking for third‑party services.

Shanbay’s Ding Yan shared their migration of all services to Docker, achieving operational efficiency, fine‑grained environment upgrades via image pulls, and leveraging Consul for service registration, health checks, and automated scaling based on CPU/memory metrics.

The discussion then covered networking choices (VLAN vs. VXLAN vs. BGP), the decision to avoid Kubernetes auto‑scaling due to performance concerns, and the use of host networking to mitigate iptables‑induced memory pressure.

Cross‑data‑center challenges were addressed, with solutions such as injecting host IPs via environment variables, using Consul for service registration, and exploring Calico for VLAN‑based connectivity.

Auto‑scaling strategies were compared: Hujiang’s Mesos+Marathon monitors aggregate CPU usage to trigger scaling, while Shanbay relies on response‑time thresholds; both note the complexities of port conflicts and the limitations of Cgroup‑based scaling.

Storage considerations were examined, contrasting Kubernetes’s volume abstraction with Docker’s storage drivers (NFS, Ceph), and discussing stateful versus stateless container deployments, including the use of PetSet‑like constructs for pod stability.

Overall, the roundtable provided a comprehensive, experience‑driven overview of container adoption, orchestration tool selection, networking, storage, and scaling practices in large‑scale production environments.