Tagged articles
big data platform
24 articles · Page 1 of 1
This article details a three-stage big data pipeline—collection, computation, and application—used by a ride-hailing platform to build user and vehicle profiles, enable intelligent order matching that boosts click-through rates by 30%, and create a self-reinforcing data flywheel that continuously improves service quality.
Architecture DesignSparkbig data platform
0 likes · 8 min read

The article compares five data‑architecture concepts—data warehouse, big data platform, data lake, data middle platform, and lake‑warehouse integration—explaining the specific problems each solves, their core characteristics, advantages, risks, and guidance on when to adopt each solution.
Lake‑Warehouse Integrationbig data platformdata architecture
0 likes · 12 min read

This article systematically explains the definitions, architectures, advantages, disadvantages, and evolution of data warehouses, big‑data platforms, data lakes, data middle platforms, and lake‑warehouse integration, helping practitioners choose the right terminology and technology for their projects.
FineDataLinkLake‑Warehouse Integrationbig data platform
0 likes · 12 min read

This article explains Bilibili's large‑scale data platform architecture, the common offline‑task failures and slowdowns users encounter, and how a large language model‑driven intelligent assistant is being built to automatically diagnose and resolve these engineering problems.
AI assistantBilibilibig data platform
0 likes · 4 min read

This article explains Bilibili's large‑language‑model‑based intelligent assistant, describing the platform's five‑layer architecture, massive daily task load, common failure and slowdown causes, and the need for AI‑driven troubleshooting to improve reliability and performance.
BilibiliIntelligent Assistantbig data platform
0 likes · 4 min read

This article explains how Bilibili’s massive video platform built a five‑layer, storage‑compute separated big‑data infrastructure and employed a large language model‑driven intelligent assistant to automatically diagnose and resolve frequent offline task failures and slowdowns, addressing common user queries about task reliability and performance.
Intelligent Assistantbig data platformlarge language model
0 likes · 4 min read

This article explains Bilibili's massive data platform architecture, the common offline‑task failures and slowdowns users encounter, and how the company applies a large‑language‑model‑driven intelligent assistant to diagnose and resolve these engineering problems efficiently.
AI assistanceBilibilibig data platform
0 likes · 4 min read

This article explains how Bilibili's massive video platform leverages a large‑language‑model‑driven assistant to diagnose and resolve offline task failures and slowdowns within its five‑layer, storage‑compute separated big data architecture, improving operational efficiency for thousands of daily queries.
AI assistanceBilibilibig data platform
0 likes · 4 min read

This article explains Bilibili's massive data platform architecture, the common reasons offline tasks fail or slow down, and how the company is exploring large‑language‑model‑driven assistants to automatically diagnose and resolve these engineering issues.
AI assistanceBilibilibig data platform
0 likes · 4 min read

This article details Bilibili's implementation of a large‑language‑model‑driven intelligent assistant that helps engineers diagnose and resolve massive offline and real‑time data‑processing failures, describing the platform’s five‑layer architecture, common failure and slowdown causes, and the need for AI‑powered troubleshooting support.
BilibiliIntelligent Assistantbig data platform
0 likes · 4 min read

This article explores Bilibili’s implementation of a large‑language‑model‑driven intelligent assistant that helps troubleshoot massive offline and real‑time data processing tasks, detailing the platform’s five‑layer architecture, common failure causes, and how AI can streamline issue resolution.
AI operationsIntelligent Assistantbig data platform
0 likes · 4 min read

This article explains how Bilibili leverages a large‑language‑model‑based intelligent agent to diagnose and resolve failures and slowdowns in its massive big‑data platform, detailing the platform architecture, workload scale, common user issues, and the need for automated assistance.
AI operationsBilibiliIntelligent Assistant
0 likes · 5 min read

WeddingJoy, a comprehensive wedding service platform, transformed its data infrastructure by adopting Alibaba Cloud DataWorks, addressing challenges of massive heterogeneous data, improving data governance, enabling AI-driven insights, and boosting development efficiency across its ecosystem.
AI integrationDataWorksbig data platform
0 likes · 8 min read

Bilibili’s one‑stop Big Data Cluster Management Platform (BMR) consolidates HDFS, Spark, Flink, ClickHouse, Kafka and other services into a unified system that evolved through four stages—standardization, metadata‑driven construction, containerization, and observability—addressing node consistency, scaling, fault self‑healing, and resource optimization while delivering elastic scaling, automated start/stop, and future cost‑saving and stability enhancements.
ContainerizationResource Optimizationbig data platform
0 likes · 12 min read

This article details Bilibili's five‑year development of a comprehensive big‑data governance platform, covering its usage scenarios, product positioning, data map and governance solutions, abstract configuration approach, operational mechanisms, and future plans, highlighting significant improvements in data efficiency and value assessment.
Bilibilibig data platformdata governance
0 likes · 10 min read

The article details how iQIYI's Magic Mirror platform evolved from a simple single‑table reporting tool to a multi‑engine, self‑service big data analysis system that improves data access speed, reduces operational costs, and supports comprehensive business analytics across the company.
Data EngineeringMagic Mirrorbig data platform
0 likes · 17 min read

This article explains Xiaomi's approach to isolating production environments for data development, covering the evolution of its data platform, the trade‑offs between physical and logical isolation, the productized workflow and security measures, and real‑world outcomes from the deployment.
big data platformdata isolationdata security
0 likes · 18 min read

The article presents an in‑depth overview of Tencent's Oula data governance platform, describing its construction goals, core capabilities, DataOps‑driven development workflow, unified metric store, data map services, and practical Q&A on asset health scoring and data lineage, illustrating a comprehensive end‑to‑end big‑data governance solution.
DataOpsTencentbig data platform
0 likes · 17 min read

The article traces the evolution of data platforms from early data warehouses—using schema‑on‑write, columnar storage, and MPP engines—to data lakes that retain raw data with schema‑on‑read, and finally to lakehouse architectures that merge storage and compute, offering unified metadata, versioning, and support for BI, big‑data, AI, and HPC workloads.
OLAPStorage-Compute Separationbig data platform
0 likes · 25 min read

NetEase’s senior big‑data engineer shares how the company’s large‑scale data platform leverages Hadoop, HDFS, YARN and related technologies, detailing multi‑layer architecture, cross‑cloud deployment, storage optimizations, NameNode performance enhancements, RPC prioritization, and practical lessons from operating petabyte‑scale clusters.
Cluster OptimizationHDFSPerformance Tuning
0 likes · 23 min read

This article details OPPO’s real‑time computing platform, covering its business scope, big‑data architecture built on Flink, Spark and Trino, the end‑to‑end job development lifecycle, SQL IDE features, diagnostic and monitoring mechanisms, link latency tracking, SLA guarantees, practical use cases, and upcoming lakehouse and cloud‑native evolution.
FlinkReal-time Computingbig data platform
0 likes · 23 min read

This article introduces a technical conference on AI engineering architecture practices hosted by Beike's AI Technology Center, featuring expert speakers and detailed agenda covering big data platforms, OLAP systems, DMP platforms, recommendation systems, and commercialization algorithms.
AI ConferenceEngineering Architecturebig data platform
0 likes · 4 min read

This article examines the technical evolution of Alibaba's DataWorks platform, detailing the pain points of a monolithic IDE, the transition through componentization, and the adoption of micro‑frontend and plugin architectures to achieve flexible, scalable, and maintainable big‑data development tooling.
Frontend ArchitectureIDEbig data platform
0 likes · 29 min read

This article presents an in‑depth overview of Beike's four‑layer streaming data platform, covering the foundational infrastructure, capability aggregation, data content, and output layers, as well as the challenges of metadata management, real‑time processing, and productization through the Ark and Tianyan systems.
Ark platformBeikeTianyan
0 likes · 14 min read
