How SF Tech Cut 10,000 CPU Cores with Apache Gluten – A Deep Dive

This article details how SF Technology adopted Apache Gluten with Velox to accelerate Spark queries, describing the architecture, task lifecycle, management framework, simulation system, unified SQL, fallback mechanisms, dynamic memory tuning, columnar shuffle, and future plans that together saved over 10,000 CPU cores and reduced operator fallback rates to around 4%.

Apache SparkGlutenVelox

0 likes · 16 min read

How SF Tech Cut 10,000 CPU Cores with Apache Gluten – A Deep Dive

DataFunSummit

Sep 21, 2025 · Big Data

Breaking the CPU Wall: BIGO’s Gluten Engine Accelerates Spark and Flink

When big‑data workloads hit the CPU wall, BIGO’s adoption of the open‑source Gluten project delivers native‑engine execution for Spark and a roadmap for Flink, achieving up to 30% end‑to‑end speedup, 50% memory savings, and a scalable, cost‑effective data processing platform.

Big DataFlinkGluten

0 likes · 16 min read

Breaking the CPU Wall: BIGO’s Gluten Engine Accelerates Spark and Flink

DataFunSummit

Aug 17, 2024 · Big Data

AnalyticDB Spark Architecture and Vectorized Engine Performance Overview

This article introduces the AnalyticDB Spark architecture, explains the need for Spark vectorization, surveys industry vectorized solutions, details ADB Spark's own vectorized implementation with Gluten and Velox, and presents performance test results showing a 6.98‑fold speedup over open‑source Spark.

AnalyticDBBig DataGluten

0 likes · 9 min read

AnalyticDB Spark Architecture and Vectorized Engine Performance Overview

Meituan Technology Team

Jun 20, 2024 · Big Data

Vectorized Execution in Apache Spark: Meituan’s Practice with Gluten and Velox

Meituan enhances Apache Spark by integrating the Gluten‑Velox vectorized execution engine, converting row‑wise operations to columnar SIMD processing, which yields over 40 % memory savings and up to 13 % faster runtimes across thousands of ETL jobs, while addressing stability, ORC support, shuffle redesign, and off‑heap memory optimization.

Apache SparkBig DataGluten

0 likes · 30 min read

Vectorized Execution in Apache Spark: Meituan’s Practice with Gluten and Velox

DataFunSummit

Mar 29, 2023 · Big Data

Gluten Vectorized Engine: Boosting Spark Performance with Native Execution

The article introduces the Gluten vectorized engine, explains why Spark’s CPU bottleneck motivates integrating native vectorized back‑ends via Substrait, details its architecture, component design, current performance gains of up to three‑fold, and outlines ongoing development and future work.

GlutenNative EngineSpark

0 likes · 18 min read

Gluten Vectorized Engine: Boosting Spark Performance with Native Execution