How Bilibili Uses LLM‑Powered Assistants to Tackle Big‑Data Task Failures

Background

Bilibili is a video‑sharing platform with massive data; its big‑data platform supports AI, commerce and other critical services.

Overall Architecture and Scale

The platform follows a “five‑layer integrated” plus “storage‑compute separation” design: a distributed file system at the bottom, an intelligent scheduling layer, compute engines such as Spark and Flink, client tools, real‑time streams (Kafka), an OLAP engine (ClickHouse), and custom tools and CI/CD pipelines.

It processes roughly 270,000 offline tasks daily, about 20,000 ad‑hoc queries, and 7,000 critical real‑time jobs. The support team receives thousands of tickets each week, with each sub‑team handling about three person‑days of inquiries.

User Problems

Users mainly ask why offline tasks fail or become slower.

Why tasks fail

Kernel defects, especially after untested upgrades.

Issues in dependent components; bugs or upgrades in shared resources cause cascade failures.

Data‑quality problems.

Other factors such as memory limits.

Why tasks become slower

Hardware aging, e.g., disk wear affecting read/write speed.

Resource scheduling pressure under massive load and mixed‑deployment policies.

Data skew or problematic data distribution.

Diagnosing these causes is labor‑intensive, prompting the exploration of intelligent, LLM‑driven assistance.

Nature of User Queries

Queries are typically terse, often just a problem description with a link or screenshot.