The ByteDance Data Platform team empowers all ByteDance business lines by lowering data‑application barriers, aiming to build data‑driven intelligent enterprises, enable digital transformation across industries, and create greater social value. Internally it supports most ByteDance units; externally it delivers data‑intelligence products under the Volcano Engine brand to enterprise customers.
ByteDance’s StreamShield delivers a three‑layer resiliency framework—engine self‑healing, hybrid replication at the cluster level, and chaos‑tested releases—that enables over 70,000 concurrent Flink jobs on 11 million CPU cores to meet strict SLAs with second‑level startup and robust fault tolerance.
In the era of rapidly advancing large‑model technology, the article outlines the challenges of evaluating data‑centric LLM agents, proposes a three‑layer evaluation framework covering basic capabilities, component‑level checks, and end‑to‑end business impact, and shares practical innovations such as semantic‑equivalence SQL matching, agent‑as‑judge pipelines, and a unified assessment platform.
This article details a scalable, distributed pipeline that uses LAS AI Data Lake, Daft on Ray, and advanced video‑processing techniques—scene detection, splitting, frame sampling, filtering, and caption generation—to transform tens of millions of hours of robot‑captured video into high‑quality, searchable semantic data while dramatically boosting CPU and GPU utilization.
The article explores how Volcano Engine’s multimodal data lake architecture addresses the storage, compute, and management challenges of AI agents by introducing new formats like Lance, upgrading engines such as Spark and Daft, and providing unified tools for processing, versioning, and querying massive multimodal datasets.
In this talk, Volcano Engine’s technical expert Chen Shuo explains the evolution of the Data Agent platform, the four‑quadrant framework for AI‑driven analytics, real‑world deployment challenges, architectural upgrades from pipeline to intelligent scheduling, and key lessons for building reliable, enterprise‑grade AI agents.
This article explores how the LAS team's AI‑driven data lake solution, built on Daft for lake computing and Lance for lake storage, tackles the emerging challenges of multimodal data handling, offering faster I/O, heterogeneous CPU‑GPU scheduling, and seamless integration for AI workloads.
In a slow‑growth economy, Haidilao’s 30‑year‑old digital transformation—driven by people‑centric change, strategic data investment, and AI‑powered analytics—enables continuous cost reduction, innovation‑led growth, and ultra‑responsive, personalized customer experiences.
Enterprises face a sharp paradox: despite exploding data volumes, only about 20% of structured data is used while the remaining 80% of unstructured data stays frozen, and this talk explores how Data Agent‑powered imperfect AI can awaken that hidden value.
ByteHouse’s BlendHouse, a cloud‑native vector database system presented at ICDE 2025, won the Best Industry and Application Paper Award, showcasing a high‑performance, universally designed framework with deep mixed‑query optimization that outperforms dedicated vector databases in read/write speed and supports large‑scale multimodal retrieval.
ByteDance Data Platform’s recent breakthroughs in database research—spanning query‑level Bayesian tuning, adaptive stream‑processing parallelism, and learned cardinality estimation—were highlighted by two papers accepted at VLDB 2025 and ICDE 2025, showcasing significant performance gains and real‑world deployments.
