Shopee's Green Computing Practices: Optimizing Resource Utilization in Data Centers

This article presents Shopee's green computing practices aimed at reducing carbon emissions in data centers. The authors detail three key technologies implemented in 2021: Overcommit (resource oversubscription), Colocation (mixed deployment), and Auto Scaling (elastic scaling).

The Overcommit strategy leverages Shopee's global user distribution across multiple time zones to schedule containers based on resource profiles without affecting service SLOs. The implementation uses an Overcommit Factor to reduce container requests while maintaining limits, combined with a custom scheduler plugin and Cgroup Agent Daemon to prevent resource contention during peak times. The system categorizes containers into Guaranteed, Burstable, and BestEffort types, and uses UsageClass concepts to manage resource allocation across different time zones.

The Colocation approach addresses Shopee's two business models: online services (latency-sensitive, high SLA requirements) running on Kubernetes and offline tasks (flexible deadline, fault-tolerant) using YARN. The mixed deployment is implemented at both cluster and machine levels, with cluster-level switching between online/offline modes and machine-level resource isolation using modified Linux CFS schedulers to prioritize online services.

The Auto Scaling solution improves upon native Kubernetes HPA by incorporating additional data sources including Statistics Platform, Pilot Portal, and Stress Test Platform. The system considers global information like inter-rack bandwidth usage and recommends optimal node matching to reduce resource fragmentation.

According to statistics, these technologies have saved thousands of machines, reducing CO2 emissions by over 40,000 tons annually based on local grid emission factors. The article concludes by inviting readers to join Shopee's efforts in building greener data centers.