This article details Xunlei's migration from a fixed Hadoop cluster to Alibaba Cloud EMR Serverless Spark, outlining the platform's background, pain points, technical upgrade goals, serverless capabilities, archive data access methods, Kyuubi integration, and the resulting business and technical benefits.
The EMR AI Assistant is an intelligent tool from Alibaba Cloud that streamlines EMR cluster management by offering quick resource queries, operation invocation, anomaly diagnosis, and conversational support, with detailed usage steps, policy code, and practical examples for big‑data operations.
Since its 2016 launch, Alibaba Cloud EMR has transformed from a basic open‑source Hadoop service into a high‑performance, AI‑enabled big‑data platform, delivering optimized I/O, vectorized processing, and integrated AI functions such as natural‑language SQL, StarRocks and Spark enhancements, while supporting diverse industry workloads.
Baiguang Technology rebuilt its data platform on Alibaba Cloud EMR, leveraging OSS storage, open‑source ecosystems, and custom elastic scheduling to handle massive, complex data workloads, resulting in up to 70% faster cluster operations, 68% lower compute costs, and significant business performance gains.
Alibaba Cloud’s open‑source EMR platform on ECS introduces managed elastic scaling that automatically adjusts task node counts, delivering up to 85% resource utilization and up to 60% cost savings across varied workload patterns, while simplifying configuration compared to custom scaling rules.
This article provides a comprehensive overview of open‑source OLAP technologies, examines various business scenarios and data‑lake architectures, and details StarRocks' core features, performance optimizations, and future development plans within the EMR ecosystem.
This article summarizes Li Yu's presentation on Alibaba Cloud EMR's deep collaboration with the StarRocks community, detailing major contributions across versions, the serverless StarRocks product’s core capabilities, and future plans to enhance OLAP‑lakehouse integration, performance, and cloud‑native elasticity.
This article introduces Volcano EMR's cloud‑native OLAP solution, detailing its product overview, storage‑compute separation, elastic scaling, cost and hot‑cold data management, intelligent query analysis, multiple customer case studies, and future roadmap for real‑time and offline data warehousing.
This article outlines Alibaba Cloud EMR's three‑stage evolution—compatibility, contribution, and beyond open source—detailing its early Hadoop adoption, Flink and Spark innovations, cloud‑native optimizations, and enterprise‑grade features such as Remote Shuffle Service, performance benchmarks, and integrated diagnostics.
This article summarizes Alibaba Cloud senior product expert He Yuan's presentation on EMR 2.0, outlining the challenges of open‑source big data, the evolution of EMR, and the new features—including cloud‑native architecture, enhanced performance, diverse resource models, and expanded analysis scenarios—aimed at reducing cost and complexity.
This article explains the concept of transient clusters and the Stateless architecture in Volcano Engine's EMR platform, compares Stateless with traditional Stateful approaches, outlines its evolution, core components, elastic scaling features, and the business value of cost‑effective, on‑demand big‑data processing.
This article explains the shift of big data and machine‑learning workloads toward storage‑compute separation and cloud‑native architectures, outlines the technical challenges of running Spark on Kubernetes, and details the EMR on ACK solution with its architecture, performance gains, and real‑world adoption.
This article explains how Alibaba Cloud EMR StarRocks extends data lake analytics to support Hive, Iceberg, and Hudi, detailing its architecture, Iceberg integration, performance gains over Trino, IO merging, lazy materialization, intelligent caching, and elastic compute capabilities for faster, unified, and cost‑effective queries.
This article explains how to design and implement a cloud‑native Lakehouse using StarRocks and Tencent Cloud EMR, covering core technical requirements, a five‑layer architecture, data ingestion with Iceberg/Hudi, performance tricks like Z‑order clustering, cost‑control through elastic scaling, and the key product features of EMR StarRocks.
This article explores the open‑source lakehouse trend, outlines the architectural features of Volcano Engine EMR, examines key challenges of building enterprise‑grade data lakehouses, and presents best‑practice case studies demonstrating how EMR enables scalable, real‑time analytics, storage‑compute separation, and seamless integration with modern big‑data engines.
This article introduces Delta Lake as an open‑source lakehouse storage framework, explains its core features, file and metadata structures, details Alibaba Cloud EMR's enhancements and deep integration with DLF, and presents G‑SCD and CDC solutions for real‑time incremental data warehousing.
This article introduces Delta Lake as an open‑source storage layer for lake‑house architectures, explains its key features, file and metadata structures, and details how Alibaba Cloud EMR and Data Lake Formation integrate and extend Delta Lake with advanced capabilities such as G‑SCD, CDC, performance optimizations, and future roadmap.
Delta Lake, an open‑source storage layer from Databricks, provides ACID transactions, data versioning, schema evolution, and unified batch‑stream processing, with a detailed file structure and metadata mechanism, while Alibaba Cloud EMR enhances it with advanced DML, performance optimizations, deep DLF integration, and solutions for G‑SCD and CDC.
ByteDance’s EMR team details how they integrated data‑lake engines such as Hudi and Iceberg into SparkSQL, streamlined jar management, built a custom Spark SQL Server with Hive compatibility, multi‑tenant support, engine pre‑warming, and transaction capabilities, dramatically improving performance and resource efficiency for enterprise workloads.
This article details how Sohu Changyou’s data team, together with Tencent Cloud engineers, planned and executed a seamless migration of over one petabyte of game data to Elastic MapReduce, Elasticsearch Service and Oceanus, achieving zero service impact and dramatically improving analytics performance.
This article describes how Soul's data engineering team replaced nightly batch ETL with real-time Delta Lake ingestion on EMR, detailing the motivations, comparative analysis of Delta, Hudi, Iceberg, the implementation architecture, encountered issues such as data skew and schema evolution, and the solutions adopted to improve performance and reliability.
This article describes how FluentSpeak replaced a DataX master‑slave pipeline with a CDC‑plus‑Delta Lake solution on Alibaba Cloud EMR, detailing architecture choices, streaming SQL merge logic, monitoring, challenges, and the resulting cost and latency improvements.
This article analyzes the challenges of traditional monolithic big‑data architectures, explains how Tencent Cloud EMR integrates Alluxio for compute‑storage separation, presents detailed performance benchmarks showing 20‑50% bandwidth reduction and 5‑40% query speedup, and outlines the specific tuning measures applied.
The article explains how Alibaba Cloud's Data Lake Building Service enables fine‑grained lake management by integrating multiple compute engines—including EMR, MaxCompute, Blink, Hologres, PAI, and open‑source Hive, Spark, and Presto—through unified metadata and OSS storage, while outlining current features, special format support, and planned future enhancements.
By parallelizing Spark’s driver‑side commit, trash, and move phases—previously single‑threaded operations that caused costly copy‑on‑rename when writing massive files to object storage—the Tencent Cloud EMR case achieved over a tenfold (1,100 %) speedup, making object storage a viable alternative to HDFS.
This article traces the 20‑year evolution of big‑data technologies, compares data lakes and data warehouses, explains their complementary strengths, and presents Alibaba Cloud’s lakehouse solution that unifies storage and compute to deliver flexible, performant, and cost‑effective analytics for enterprises.
This article examines the challenges of running Spark on Kubernetes, introduces the Remote Shuffle Service architecture to overcome shuffle bottlenecks, details EMR on ACK integration, showcases performance gains with Terasort benchmarks, and outlines future cloud‑native big‑data strategies such as mixed‑cluster and serverless deployments.
Huya migrated its petabyte‑scale Hadoop workloads to Tencent Cloud EMR, using a dedicated line and COS for warm/cold storage, enabling minute‑level cluster provisioning, rapid analysis, and up to 60 % cost savings while improving flexibility, efficiency, and data‑driven innovation.
The article details a scalable, extensible cloud‑audit architecture for Tencent Cloud accounts that stores API logs in a Shanghai‑region COS bucket, processes them with EMR‑based Hive tables and hourly partition scripts, aggregates results into a hot MySQL store, and enables administrators to monitor all sub‑accounts with a real‑time “god view.”
