Huya Live Streaming Big Data Practice: Cloud‑Based Data Platform and EMR Solution

Introduction Huya, the first listed Chinese game‑streaming company, operates platforms such as Huya Live and NimoTV, serving over 150 million monthly active users across PC, web and mobile. To connect high‑quality content with users more intelligently and efficiently, Huya’s big‑data team sought a cloud‑native solution and partnered with Tencent Cloud EMR.

1. Background The company’s massive user base generates petabyte‑scale data. Traditional on‑premise IDC clusters faced two main problems: slow response to urgent analysis tasks and high cost of maintaining idle hardware.

2. Big‑Data Scenario Huya’s team, built on an open Hadoop stack, supports offline and real‑time processing of hundreds of petabytes, enabling content recommendation, operation analysis, and user‑experience improvement.

3. Challenges

• Timeliness : Sudden analysis jobs could not be accommodated quickly because adding new hardware to the IDC cluster required weeks.

• Cost : Cold data stored on expensive hardware reduced utilization, increasing storage costs.

4. Cloud‑Based Big‑Data Solution The solution integrates Huya’s IDC network with Tencent Cloud via a dedicated line, migrates warm/cold data to Tencent Cloud Object Storage (COS), and creates on‑demand Hadoop clusters with Tencent Cloud EMR. For a 2019‑year‑wide analysis task, the EMR cluster with hundreds of nodes was provisioned in about 10 minutes, completed the job within two hours, and delivered results in 1.5 days. After completion, the cluster was terminated to avoid ongoing costs, while COS retained the data for future use.

5. Core Value of Cloud Migration

• Flexibility & Efficiency : Minute‑level cluster creation and automatic teardown provide rapid scaling.

• Cost Savings : Using COS for warm/cold storage replaces costly IDC hardware, and on‑demand EMR clusters eliminate the need for permanently provisioned resources, achieving up to 60 % flexible cost reduction.

6. Advantages of Cloud Data Architecture Cloud providers offer a rich ecosystem of big‑data services covering infrastructure, toolchains, and domain‑specific applications. Enterprises can quickly migrate or integrate existing data architectures into the cloud, benefiting from massive storage/computing capacity and continuous innovation from cloud vendors.

The Huya‑Tencent Cloud EMR collaboration exemplifies the industry trend toward cloud‑native big‑data platforms that drive data‑driven business and operational innovation.