Challenges and Optimization Strategies for Containerized Deployment of Online Services on Kubernetes

Tencent has widely applied cloud‑native containerization based on Kubernetes (K8s) across its massive online services. The migration from traditional VM deployment to container deployment introduced several transformation challenges, including container delivery, node balancing, K8s cloud‑native features, and cluster pooling.

Key challenges: each Pod must declare its resource size, and changing the size requires destroying and recreating the Pod, making mixed‑deployment difficult; nodes host heterogeneous Pods, making node balancing a challenge; K8s elasticity must meet the production demands of online services; and mapping tens of thousands of services to clusters requires effective cluster pooling.

Optimization measures:

Resource utilization improvement – dynamic compression and over‑commit: compress CPU requests while keeping limits unchanged, and dynamically over‑sell CPU based on load.

Node load balancing – dynamic scheduling and rescheduling: a custom scheduler senses real‑time node load, detects high‑load Pods via Problem Detectors, and re‑schedules sensitive Pods to idle nodes.

K8s elastic scaling – collaborative scaling: enhanced HPAPlus‑Controller for custom scaling policies and VPAPlus‑Controller for rapid scaling of bursty workloads, including stateful services.

Cluster resource management – dynamic quota and resource migration: an Operator controls service visibility and quota, while a dynamic planning operator reallocates nodes across clusters during peak events.

Dynamic routing challenges: container destruction/recreation triggers routing updates; frequent container changes, massive scale, and mixed cloud‑on‑prem routing increase complexity. Solutions include a unified routing‑sync controller, Service‑level event aggregation, dual‑queue models separating real‑time and periodic events, and a master‑standby controller architecture with fast failover.

Self‑healing mechanism for container destruction failures: a new self‑healing workflow detects and resolves stuck container deletions, integrates dynamic scheduling, elastic scaling, and disaster‑recovery migrations, and employs a decision‑tree model (using information entropy, Gini coefficient, pruning) to quickly locate root causes and automate remediation.

The article concludes with a step‑by‑step guide to building and optimizing the decision‑tree model for intelligent operation and maintenance, aiming to improve fault localization efficiency and provide a closed‑loop solution for large‑scale container failures.