How Bilibili Leverages Large Language Models to Automate Big Data Task Troubleshooting
This article explains Bilibili's massive big‑data platform architecture, the common offline‑task failures and slowdowns users encounter, and how a large‑language‑model‑driven intelligent assistant is being built to diagnose and resolve these issues automatically.
Background
Bilibili is a video‑sharing platform with massive data volumes; its big‑data platform supports many business lines such as AI and commerce. The platform follows a "five‑layer integrated" plus "storage‑compute separation" architecture: a distributed file system at the bottom, an intelligent scheduling layer, various compute engines (Spark, Flink, etc.), client tools, real‑time streams (Kafka), OLAP engines (ClickHouse), and custom CI/CD tools.
The platform processes an enormous workload: about 270,000 offline tasks daily, roughly 20,000 ad‑hoc queries, and around 7,000 critical real‑time jobs. Support teams handle thousands of weekly inquiries, each small team spending about three person‑days per week answering questions about task failures or slowdowns.
User Issues
Why tasks fail
Kernel defects – upgrades without sufficient testing can cause large‑scale failures.
Dependency component problems – upgrades or bugs in heavily used components break dependent tasks.
Data quality issues – corrupted or malformed data leads to failures.
Other factors such as memory errors may also contribute.
Why tasks slow down
Hardware aging – massive storage fleets experience wear, reducing read/write speeds over time.
Resource scheduling pressure – high user volume and mixed‑deployment policies cause contention and throttling.
Data distribution problems – data skew or inherent data issues increase processing time.
Because diagnosing these diverse failure and slowdown causes is time‑consuming, Bilibili is exploring intelligent, AI‑driven methods to assist users in pinpointing and resolving problems automatically.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
DataFunTalk
Dedicated to sharing and discussing big data and AI technology applications, aiming to empower a million data scientists. Regularly hosts live tech talks and curates articles on big data, recommendation/search algorithms, advertising algorithms, NLP, intelligent risk control, autonomous driving, and machine learning/deep learning.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.