tide power

Baidu Geek Talk

Jun 7, 2023 · Big Data

Optimization Practices for Offline Big Data Computing and Storage at Baidu MEG

Baidu MEG’s offline big‑data platform cut costs and boost efficiency by applying intelligent scheduling, storage‑separation, tide‑power workload profiling, remote shuffle services and dynamic quota resizing, raising compute utilization from 55 % to 80 % and storage from 63 % to 78 %, slashing annual expenses by roughly ¥70 million and reducing task duration by about 30 %.