How MaxQA Supercharges Query Performance for Large‑Scale Data Warehouses

This article is the fifth part of a series that follows the real migration journey of a leading Southeast Asian technology group (referred to as GoTerra) from Google Cloud Platform's BigQuery to Alibaba Cloud MaxCompute, focusing on the performance optimization techniques required to match BigQuery.

Business Background and Pain Points

GoTerra operates ride‑hailing, e‑commerce, food delivery, logistics, and financial payment services. Before moving to MaxCompute, it relied on BigQuery, which offered strong SQL compatibility, automated migration, streaming writes, metadata upgrades, and intelligent resource scheduling. Three high‑latency‑sensitive scenarios drive the need for faster queries:

BI reporting: business users require query results within 30 seconds.

Ad‑hoc customer‑service queries: real‑time support agents need sub‑5‑second responses.

Data pipelines: core business processes enforce a 30‑second timeout for automated jobs.

These scenarios demand that MaxCompute meet or exceed BigQuery's performance and stability.

Product Advantages – MaxQA Accelerator

MaxQA (MaxCompute Query Accelerator) introduces a dedicated control layer and a query‑compute resource pool, delivering lower latency, higher concurrency, and stronger stability. It remains fully compatible with the MC SQL feature set, allowing existing SQL jobs to migrate without code changes while improving performance and reducing overall cost through flexible resource‑time‑sharing policies.

Technical Solution Overview

The overall MaxQA architecture consists of the following core modules (from top to bottom):

BI layer – multiple BI tools compatible with MaxQA (see MaxCompute BI analysis link).

Control layer – includes TopConsole for project, quota, and tenant management, and DataWorks for scheduling temporary or periodic queries.

User interface and access layer – clients can use Java, Python, Go SDKs or JDBC to connect to a MaxQA instance.

MaxQA instance layer – composed of a control layer, compute layer, and shared storage layer.

Inside a MaxQA instance:

Coordinator : entry point that contains the API layer, SQLTask (SQL parsing, plan generation, permission checks, source‑table split, operator codegen), and JobDriver (drives job execution based on the plan).

Pre‑warmed Worker pool : a set of Worker processes started on each machine at instance creation, each embedding the SQL execution engine and storage engine.

MaxQAAdmin : maintains the Worker pool and handles seamless hot upgrades.

ShuffleAgent : transfers shuffle data between upstream and downstream tasks.

CacheAgent : caches intermediate results or hot data to reduce storage access.

MaxQA Usage – Quota Model

MaxCompute quotas are organized into primary (level‑1) and secondary (level‑2) quotas.

Pay‑as‑you‑go (post‑paid): billed by actual usage, suitable for bursty workloads.

Subscription (pre‑paid): fixed‑duration purchase, lower cost for stable workloads.

Secondary quotas include batch‑type (periodic large‑scale jobs) and interactive‑type (BI reports, ad‑hoc queries). Job submission flows differ between batch and interactive quotas, as illustrated in the diagrams.

Core Optimization Points

Dedicated control layer and isolated compute pool : each interactive quota gets its own MaxQA instance, eliminating multi‑tenant interference.

Direct query routing : FrontEnd forwards queries straight to the instance’s Coordinator, shortening the submission path.

Simplified protocol : queries under 5 seconds and <10 MB result sets complete in a single request.

Asynchronous metadata writes : InstanceMeta/TaskMeta writes are performed asynchronously, and non‑critical post‑execution steps (e.g., summary generation) run in the background.

Engine optimizations :

Control layer enhancements :

Full‑link caching :

Flexible Elastic Resources

Timed scaling and elastic CU allow dynamic allocation of compute units based on workload peaks, improving utilization and reducing idle costs. AutoScaling further detects load spikes in real time and automatically expands CU resources, charging only for actual usage.

Business Value

MaxQA enabled GoTerra to migrate smoothly from BigQuery to MaxCompute with zero user impact, maintaining continuous service. After adoption, query efficiency doubled, confirming MaxQA’s high‑performance, low‑latency, and high‑stability advantages for large‑scale data workloads.

Future Outlook

Pipeline‑style multi‑threaded engine upgrades for higher throughput.

Enhanced stability through automatic anomaly detection, isolation, and transparent fallback to Serverless pools.

Intelligent data pre‑heating based on historical job patterns.

Improved shuffle‑data cache identification accuracy.

These continuous improvements aim to deliver faster, more reliable, and smarter one‑stop big‑data query experiences.