Apr 25, 2024 · Big Data

What Happens Behind the Scenes When a SQL Query Runs in a Big Data Platform?

This article walks through the end‑to‑end lifecycle of a SQL task in a big‑data environment, covering creation, scheduling metadata, instance generation, resource allocation, ODPS execution, and final processing on the Fuxi distributed engine.

FuxiODPSSQL

0 likes · 11 min read

What Happens Behind the Scenes When a SQL Query Runs in a Big Data Platform?

Alibaba Cloud Developer

Jan 2, 2020 · Big Data

How Alibaba’s MaxCompute Tackled Double‑11’s EB‑Scale Data with Fuxi 2.0 and StreamlineX

In 2019 Alibaba’s MaxCompute processed near‑exabyte daily data during Double 11, using the newly released Fuxi 2.0 scheduler, StreamlineX + Shuffle Service, and the upgraded DAG 2.0 engine to overcome massive throughput, resource‑allocation, and fault‑tolerance challenges while achieving significant performance and stability gains.

DAG 2.0FuxiMaxCompute

0 likes · 28 min read

How Alibaba’s MaxCompute Tackled Double‑11’s EB‑Scale Data with Fuxi 2.0 and StreamlineX