This whitepaper defines AI-driven autonomous data governance with a five-layer architecture, three autonomy levels, and human-in-the-loop boundaries, detailing implementation scenarios, a DataGovernance-MLOps loop, phased rollout strategies, ROI metrics, organizational changes, and a retail case study showing 93% asset coverage and 68% ticket reduction.
DACon 2026 Beijing revealed that enterprise AI agent deployment fails not from weak models but from missing formalized business ontology — unified metadata, semantic definitions, and executable logic — with speakers from JD Health, Baidu, and industrial platforms demonstrating how structured knowledge graphs enable verifiable, high-ROI agent workflows.
This article explains data lineage concepts, table-level vs field-level lineage, why major tech companies prioritize it, core use cases like impact analysis and root cause detection, three implementation approaches, common pitfalls, and a practical four-step guide for enterprises to start building lineage from critical reports.
This article outlines a comprehensive data governance framework covering organizational awareness, metadata management, security, integration, services, and visualization to transform data into a strategic asset with a phased roadmap for enterprise value realization.
Data Governancedata asset managementdata integration
DACon 2026 brings together 12 leading Chinese companies including Ant Group, Gaode, Zhihu, and Li Auto to detail how they built semantic layers that bridge LLMs and enterprise data warehouses, revealing concrete architectures, accuracy benchmarks, and lessons learned from moving beyond demos to production-grade Data Agents.
This article dismantles twelve common data governance fallacies — from demanding instant metadata completion to treating governance as a cost center — arguing that effective governance requires confronting upstream data pollution, aligning incentives across departments, and prioritizing critical data paths over blanket coverage.
This article explains the eight core modules of data governance—standards, metadata, catalog, quality, master data, assets, lineage, and security—detailing how each solves specific problems and why they must integrate into a continuous lifecycle loop rather than exist as isolated tools.
This article presents a comprehensive framework for building a big data governance platform, covering a five-layer architecture, nine core governance modules, a three-phase implementation roadmap, organizational structure, success metrics, and industry-specific selection guidance to transform chaotic data into trusted, findable, and controllable assets.
Implementation Roadmapbig data governancedata architecture
The article explains how scattered, hard‑coded metadata in large‑scale systems leads to unknown impact of changes, and walks through building a unified metadata center with a single source of truth, automatic collection, lineage, impact analysis, asset discovery, active metadata, and governance, while highlighting practical trade‑offs and real‑world tools.
This article argues that data sharing initiatives often fail not due to access permissions but because identical metric names mask divergent definitions, statistical scopes, and processing logic, a problem amplified in AI-driven workflows where semantic context must be explicitly preserved for reliable automated decisions.
AI governanceData Governancecross-domain data circulation
This article traces data governance evolution from master data consistency and metadata visibility to ontology-driven business semantic modeling, arguing that AI-era governance must produce high-quality datasets that are AI-usable, trustworthy, evaluable, and continuously optimized through explicit business object, process, rule, and evidence modeling.
The article argues that traditional master data and metadata management only achieve data visibility and traceability, while ontology-driven governance adds a computable business semantic layer—modeling objects, processes, states, rules, and actions via OPM and ontology—to enable AI agents to execute tasks reliably within real business contexts.
AI agentsBusiness Semantic GovernanceData Governance
The article breaks down data governance into four essential pillars—data standards, data quality, data security, and metadata management—illustrating why each is critical, how they interrelate, and practical steps enterprises can take to embed them into data pipelines for trustworthy, secure, and discoverable analytics.
The article analyzes why data‑governance projects still fail when combined with AI, presents a four‑layer NL2SQL architecture, details agent responsibilities, metadata‑governance methods, anomaly‑diagnosis and permission‑control flows, outlines dataset‑building stages, evaluation metrics, and provides a step‑by‑step rollout roadmap.
OPPO tackles explosive multimodal data growth by unifying metadata with Gravitino and boosting I/O performance using the open‑source Curvine cache, delivering a four‑layer data‑lake architecture that resolves data islands, metadata chaos, and bandwidth bottlenecks while achieving near‑commercial query speeds.
The article breaks down the distinct concepts of data governance, data management, and data control, detailing their purposes, core components, practical workflows, and common pitfalls so teams can align strategy, avoid resource waste, and implement effective data practices.
In a Data for AI meetup, Datastrato's VP of Engineering Shi Shaofeng explains how Apache Gravitino's metadata federation, metalake architecture, and unified access control address multi‑cloud data fragmentation, compliance, and AI‑driven governance while outlining version 1.1.0 enhancements and the roadmap for 1.2.0.
OPPO’s data‑lake team, led by David, detailed their transition from Hive‑Spark to a unified multi‑modal lake, leveraging Gravitino for cross‑engine metadata management and the open‑source Curvine cache to eliminate data silos, boost I/O performance, and support massive image, recommendation, and AI‑Agent workloads.
Most companies see data governance fail not because of technology but because they start with the wrong direction, focusing on rules, platforms, and processes that add friction instead of improving data usability, and the article provides a step‑by‑step, low‑overhead approach with concrete SQL and Python templates to fix it.
In the era of AI and multi‑cloud, this article analyzes the core challenges of data governance—data silos, quality gaps, and compliance risks—and explains how Apache Gravitino’s unified metadata architecture together with OpenLineage’s standardized lineage model provide a scalable, automated solution for intelligent, real‑time data management.
The article explains the distinct purposes, contents, usage scenarios, and target users of data catalogs and data dictionaries, illustrates their complementary relationship, and highlights practical challenges and a tool‑based approach for keeping them up to date.
This article details the industry-standard dual pattern of file body storage and metadata registration for unstructured data lake ingestion, covering a three-layer architecture, batch and real-time implementation steps using Paimon and Flink, AI-specific optimizations like label standardization and format conversion, and key operational considerations for permissions, versioning, and cost control.
This article explains the concept of Data Fabric, debunks common misconceptions, outlines the three key drivers behind its rise, and provides a practical four‑step roadmap—including metadata, semantic layers, policy engines, and AI—to help teams of any size adopt the technology.
The article explains why scattered, unprepared data hampers analysis, defines data warehouses versus traditional databases, outlines the three‑layer architecture (ODS/Staging, DWD, DWS/ADS), details the ETL process, introduces dimensional modeling, and discusses metadata, data‑quality management, and emerging cloud‑native and real‑time trends.
The article shares a practical roadmap for building a data governance framework, covering data standards, metadata management, master data management, data quality, security, and asset management, with step‑by‑step methods, real‑world examples, and tools to help teams align definitions, automate checks, and protect sensitive information.
The article outlines a data governance framework for big data platforms that addresses enterprise data standards, quality, and metadata management, detailing three core objectives—standard implementation, quality analysis, and lineage mapping—while providing a PDF roadmap for practical deployment.
The article examines common data‑warehouse pitfalls—poor data quality, endless ad‑hoc queries, and unreadable SQL—then outlines a systematic approach that includes correcting mindset, designing a layered architecture, applying disciplined modeling practices, enforcing data governance, and managing engineering details for sustainable success.
This article collection showcases seven advanced data engineering solutions—from Tencent Cloud's Iceberg batch‑stream integration and Apache Gravitino metadata lineage to Xiaohongshu's Lakehouse evolution and multimodal AI data lake implementations—highlighting architectural innovations, performance optimizations, and real‑world deployment insights for modern big‑data platforms.
The article compares data warehouses, data lakes, and lakehouse architectures, illustrating each with real‑world projects, highlighting their core concepts, strengths, limitations, key technologies, and a step‑by‑step migration path to help teams select the most suitable solution for their business needs.
This article series explores Tencent Cloud's Iceberg‑based batch‑stream integration, Apache Gravitino's unified metadata and lineage solution, Xiaohongshu's data‑architecture evolution for the Big AI Data era, and a practical Data+AI multimodal data‑lake implementation, highlighting challenges, architectural designs, and performance gains.
The article defines data lineage, illustrates its importance for rapid root‑cause analysis, impact assessment, compliance and asset management, and walks through a three‑step approach—collection, integration, validation—using concrete examples and a real‑world tool.
This article introduces Douyin Group’s Data Asset Management Platform, explaining how its focus on data assets rather than raw metadata enables a comprehensive, real‑time big‑data lineage system that supports search, AI‑driven discovery, and diverse application scenarios across the organization.
This article introduces Douyin Group’s Data Asset Management Platform, explaining its shift from traditional metadata to a comprehensive data‑asset approach, detailing the platform’s capabilities, and focusing on the evolution and application of full‑link data lineage across four key topics to improve visibility, quality, security, and cost efficiency.
This article shares a three‑year journey of building a data‑virtualization‑based, multi‑environment feature management framework for real‑time risk decision platforms, detailing challenges like heterogeneous storage, cold‑start, and operational stability, and presenting a unified architecture that decouples physical storage from business logic.
This article introduces Douyin Group’s Data Asset Management Platform, explaining its shift from traditional metadata to comprehensive data assets, detailing the evolution, architecture, and applications of its full‑link big data lineage, and offering strategic guidance for building effective lineage systems.
The Mantle system, presented in a SOSP'25 paper by Baidu's storage team and collaborators, delivers a distributed hierarchical namespace for cloud object storage that overcomes traditional scalability and performance limits, enabling massive data lake workloads with dramatically reduced latency and vastly increased throughput.
This article explains the architecture, core components, and step‑by‑step implementation of a tool usage module that enables AI agents to standardize, select, execute, and transform external tools, illustrated with a sales data analysis case and detailed code snippets.
Xiaomi’s data‑lake team explains how they tackled small‑file issues, unified metadata with Gravitino, migrated Hive to Iceberg and Fileset, leveraged JuiceFS for multi‑cloud storage, and combined Iceberg and Paimon to achieve cost‑effective, high‑performance batch and real‑time analytics.
JD Retail faces growing data volume, redundant models, and resource‑intensive storage, prompting a comprehensive data‑governance strategy that defines standards, streamlines architecture, isolates development, and optimizes compute and storage costs, ultimately enabling more efficient, secure, and agile data operations across the enterprise.
This article examines the essence of data governance, outlines its four core domains, proposes a strategic and technical implementation roadmap, evaluates effectiveness with the DCAM model, and explores how AI large models can enhance metadata, data quality, and compliance while highlighting practical limitations and future trends.
This article details Unicom Digital’s metadata management practice on its integrated data platform, covering the strategic background of data, key challenges, award-winning capabilities, three-pronged solutions—automation, linking+, and AI—along with practical implementations, full‑chain lineage, data responsibility, lifecycle management, and future AI‑driven enhancements.
Apache Gravitino is an open‑source metadata service platform that provides a unified, high‑performance, geographically distributed metadata lake, enabling end‑to‑end data governance, multi‑engine access, and direct management of both structured and unstructured data assets across diverse systems.
Kafka 4.0 introduces KRaft, a ZooKeeper‑free metadata layer built on the Raft consensus algorithm, detailing role transitions, leader election, log replication, controller and broker responsibilities, and fault‑tolerance mechanisms, enabling a more scalable and self‑managed architecture for large‑scale distributed streaming.
DeepSeek, the popular Chinese large‑language model, boosts data‑warehouse engineers' productivity by offering free, open‑source AI assistance across code writing, model design, metadata management, data quality monitoring, and governance, ultimately maximizing enterprise data asset value.
This article defines data architecture per ISO/IEC/IEEE 42010 and DCMM, outlines core components — data model, flow, metadata, integration — identifies four key challenges including siloed management and technology-first approaches, and proposes optimization paths: enterprise-wide architectural governance, layered stable design, and collaborative data governance.
The article presents HadaFS, a novel burst‑buffer‑based distributed file system that combines the scalability of local burst buffers with the data‑sharing advantages of shared buffers, details its LTA architecture, metadata handling, the Hadash management tool, and extensive performance evaluations on the SNS supercomputer.
Bilibili adopted Apache Gravitino as a unified metadata platform that decouples consumers, consolidates schemas and Fileset‑based unstructured data across heterogeneous sources, cuts metadata and storage costs, resolves inconsistencies, boosts Hive Metastore performance, and enables features such as Iceberg branching and future AI‑centric governance.
The article explains why MySQL 8 replaces fragmented, non‑transactional metadata files with a unified, InnoDB‑backed transactional data dictionary, details the information_schema views that expose it, and enumerates the resulting benefits such as atomicity, consistency, crash recovery, performance gains, simplified administration, reduced lock contention, smoother upgrades, improved DDL handling, and enhanced security.
Huolala’s data team details the evolution of its metadata management platform—covering architecture, stages from early Hive‑ETL to real‑time field‑level lineage, AI‑driven smart search, cost‑governance mechanisms, and security classifications—showcasing practical solutions for data discoverability, efficiency, and protection at scale.
Douyin Group’s data asset management platform introduces a systematic "manage, find, use" approach that unifies metadata collection, full‑coverage data lineage, and a suite of applications across development, governance, asset utilization, and security, while outlining its architecture, modeling, quality metrics, and future roadmap.
This article provides a comprehensive overview of Douyin Group's data asset management platform, detailing the evolution, architecture, and applications of its large‑scale data lineage system, and discusses future directions for enhancing data quality, cost efficiency, and security across the organization.
Magnus is Bilibili’s self‑developed intelligent service that continuously optimizes Iceberg tables by scheduling snapshot expiration, orphan‑file cleanup, manifest rewriting, and multi‑dimensional data optimizations—including small‑file merging, sorting, distribution, and index creation—while automatically recommending configurations from real‑time query logs, delivering over 99.9% task success and up to 30% scan‑data reduction.
This article presents Qichacha's comprehensive data‑cost‑reduction strategy, detailing its Hadoop‑based three‑pillar architecture, layered data warehouse, Hive upgrades, unified metadata across multi‑cloud clusters, middleware choices such as Alluxio and JuiceFS, version‑compatible hybrid clouds, and Kubernetes‑driven resource orchestration to achieve scalable, low‑cost data processing.
This article examines the challenges of Hudi metadata stored on HDFS, introduces the independently developed Hudi MetaServer for centralized metadata, visual management, unified permission control, TTL, expression payloads, and multi‑active scaling, and outlines future enhancements such as LLS, multi‑table fusion, and JDBC support.
This article introduces Tencent's Big Data Processing Suite (TBDS) and the open‑source Gravitino project, explaining how they provide a unified metadata service and a comprehensive, extensible permission model to address data and permission islands across heterogeneous Hadoop and MPP ecosystems.
Bilibili’s one‑stop Big Data Cluster Management Platform (BMR) consolidates HDFS, Spark, Flink, ClickHouse, Kafka and other services into a unified system that evolved through four stages—standardization, metadata‑driven construction, containerization, and observability—addressing node consistency, scaling, fault self‑healing, and resource optimization while delivering elastic scaling, automated start/stop, and future cost‑saving and stability enhancements.
ContainerizationResource Optimizationbig data platform
This article explains how to construct a comprehensive data indicator system by outlining its background, design, standardization, metadata management, and future applications, while addressing business, technical, and product challenges and showcasing practical examples and visual workflows.
This article presents a comprehensive data lineage system (Data-Lineage) for big data platforms, addressing challenges in heterogeneous data sources, multiple execution engines, and complex dependencies through hook-based architecture and modular design.
vivo’s CICD artifact management has evolved from manual builds to a comprehensive Platform Management 2.0 that provides unified storage, multi‑type support, version control, promotion, security scanning, lifecycle policies, and fine‑grained access, dramatically reducing errors and operational costs.
This article presents Bilibili's practical data governance operational framework, introducing the DAMA‑Bok methodology, detailing two real‑world cases on storage‑level risk and data‑loss post‑mortem, and outlining the organizational, metadata, and embedded governance mechanisms that drive cost and quality improvements.
This article introduces Tencent's Big Data Processing Suite (TBDS), discusses challenges of data silos, and presents Gravitino's open‑source unified metadata service and permission model, detailing how it integrates Hadoop, MPP, and various catalog plugins to provide consistent access control across heterogeneous data platforms.
This article describes how Tubi built the Rosetta Stone system—a flexible ID mapping workflow that leverages large language models, embedding similarity ranking, and K‑nearest‑neighbors to unify and enrich metadata across a 200,000‑title library, improve content recommendation, and streamline operations.
This article details Bilibili's comprehensive approach to standardizing event tracking (埋点), covering its definition, data pipeline, common business issues, metadata‑driven management strategies, efficiency gains, and future prospects for unified real‑time and batch processing.
This article examines Alluxio's open-source distributed file system, detailing the core types of metadata—inode, block, mount table, and worker—along with the mechanisms for their storage, management, and optimization in both HEAP and ROCKS modes, and provides practical configuration guidance for scaling large-scale data environments.
This article presents a comprehensive case study of NetEase Cloud Music’s metadata‑driven intelligent governance platform, detailing its scale, construction background, modular architecture, rule‑based automation, practical deployment, and future roadmap for sustainable data ecosystem management.
Facing fragmented metadata, unclear ownership, and costly data duplication, Weimeng implemented a comprehensive data asset governance framework—covering metadata standards, lineage visualization, metric normalization, and cost management—to boost data quality, security, and business value across its new‑retail platform.
Didi’s data serviceization converts raw business data into consumable services through a four‑stage pipeline—integration, development, production, and back‑flow—while the Data Dream Factory and Shu‑Chain platform automate synchronization, provide a unified access gateway for thousands of APIs, and introduce a standard metric service that abstracts storage complexities and ensures high‑performance, secure data delivery.
This article details the architecture, design decisions, and iterative improvements of the Data Catalog product within Volcano Engine's DataLeap suite, covering metadata management, ingestion pipelines, search optimization, lineage capabilities, storage layer enhancements, and future development directions.
This article examines the challenges of maintaining stability across cloud and on‑premise environments, explains how Ant Group's Business‑Intelligent Observability Service (BOS) addresses these issues through unified metadata, seamless application integration, data standardization, and extensive case studies, and demonstrates the resulting improvements in reliability and operational efficiency.
This article explains Alibaba Cloud's data lake architecture, unified metadata services, storage management optimizations, and format handling techniques, illustrating how lakehouse concepts, multi‑engine support, and lifecycle policies enable efficient, secure, and cost‑effective big data processing in the cloud.
This article details DataCake's data‑governance journey, covering the problems of data silos, unclear costs, and tool fragmentation, then explains the strategic thinking, the multi‑layered solution architecture, and the measurable outcomes such as higher resource utilization and reclaimed storage.
This article explores the key problems data governance aims to solve, outlines a comprehensive governance framework, and details practical implementation steps—including tool integration, metadata management, lake‑in and lake‑out processes, and governance policies—to achieve a closed‑loop, value‑driven data ecosystem.
The article outlines DataCake's data governance journey, describing the challenges of data silos and cost inefficiencies, the strategic thinking behind a unified metadata platform, the implementation of governance tools, cost analysis modules, and asset inventory, and concludes with results, future plans, and a Q&A session.
This article explains the problems that data governance addresses, outlines a comprehensive governance framework—including system architecture, processes, and policies—and describes practical implementation steps such as integrated tooling, standardized modeling, metadata management, lake‑in and lake‑out governance, and organizational structures for sustainable data management.
This article presents an in‑depth overview of DataLeap's data lineage capabilities, covering the challenges, multi‑layer model design, implementation with Apache Atlas and JanusGraph, performance optimizations, diverse use cases across asset, development, governance and security domains, and future trends for lineage technology.
Youzan built an automated offline data cost‑optimization platform that gathers accurate metadata, mines unused or failing tables and tasks, and safely decommissions them through a backend‑frontend workflow with owner validation, notifications, rollback safeguards, and plans to extend lineage coverage and real‑time asset handling.
The article explains why data is a critical corporate asset, distinguishes narrow and broad data‑governance approaches, outlines strategic principles such as treating governance as a systematic, prioritized effort, and presents eight real‑world case studies from companies like Tencent, SF Tech, Huolala, and NetEase.
This article presents Tencent's comprehensive data governance framework, covering its definition, objectives, challenges, methodology, organizational structure, metadata management, data asset lifecycle, security measures, and technical implementation details such as microservice architecture, data collection, lineage analysis, and storage solutions.
Zhihu’s real‑time computing platform, initially built as Skytree 1.0 on Kubernetes and later re‑engineered as Mipha 2.0 with Flink SQL, unified metadata management, dynamic jar loading, UDF support, Protobuf format, CDC integration, and extensive operational optimizations, now processes petabyte‑scale data with high reliability.
This article explains what data governance is, why enterprises need it, the key components such as data quality, metadata, master data, asset and security management, and provides a step‑by‑step framework, organizational structure, platform features, evaluation methods and common pitfalls.
This article details Kuaishou’s evolution of its metadata management platform—from early Hive‑centric beginnings to a unified 2.0 architecture and a forward‑looking 3.0 vision—highlighting challenges, key technologies, and how metadata drives data production, consumption, governance, and cost optimization across the big‑data middle platform.
ByteDance’s data platform team explains their real‑time data lake implementation, covering its evolving definition, six core capabilities, challenges such as data management, concurrent updates, performance and log ingestion, and detailed case studies of multi‑stage deployment, indexing, metadata services, and future roadmap.
This article explains how Ozone, the Hadoop community’s new distributed object‑storage system, overcomes HDFS’s small‑file limitations with a hierarchical Volume‑Bucket‑Object model, detailing its architecture, components, data flow for creating and reading objects, and the benefits of its scalable, fault‑tolerant design.
GooseFS, Tencent Cloud’s Hadoop‑compatible storage accelerator, adds a local NVMe‑SSD cache layer to cloud‑native data lakes, letting users boost query speeds by up to 46 % and cut backend bandwidth by 200 Gbps without code changes, as demonstrated by a music‑industry customer’s 200‑node deployment caching ten million files.
The article explains why event‑tracking data needs governance, outlines a full‑link governance methodology, describes the organizational setup, and details the features of JD Tech’s one‑stop tracking management platform, including metadata unification, one‑click validation, real‑time dashboards, visualization tools, and H5‑native data integration.
Data GovernanceH5-native integrationReal-time Dashboard
JD’s article details its big‑data platform’s cross‑domain and hierarchical storage solutions, describing the challenges of multi‑datacenter data synchronization, the architecture of its storage layer, the implemented asynchronous and synchronous data flows, topology management, metadata tagging, and performance‑enhancing techniques for efficient, disaster‑resilient data handling.
Data PlatformHierarchical Storagecross-domain storage
This article presents detailed case studies of ByteDance and Alibaba implementing real‑time data lake solutions with Hudi and Flink, describing the business drivers, architectural challenges, and the specific technical strategies such as unified metadata layers, optimistic locking, scalable hash indexing, and CDC‑based incremental ETL to achieve low‑latency, high‑throughput data processing.
This presentation explains where knowledge resides in data architectures, demonstrates knowledge‑graph‑driven skill discovery, metadata management, and semantic search, and concludes with a comparison of GraphQL and Cypher for graph queries, illustrated with real‑world Neo4j case studies.
This article details ByteDance's 2021 overhaul of its Data Catalog system, the performance regressions encountered after switching to Apache Atlas, and the step‑by‑step backend optimizations—including JanusGraph tuning, Gremlin query refactoring, parallel processing, and write‑path improvements—that reduced latency from minutes to seconds.
This article explains the origin and definition of data lakes, describes their ability to store structured, semi‑structured and unstructured data at any scale on‑premises or in the cloud, outlines essential lake capabilities such as unified storage, raw‑data preservation, scalable compute, metadata and security management, and compares data lakes with data warehouses and lakehouse architectures through real‑world cloud‑native examples.
Apache Druid manages metadata through a layered, distributed system where the Overlord coordinates ingestion tasks, MiddleManagers launch Peons to create segments, Coordinators and Historical nodes store and serve segment data, Brokers route queries, while MySQL, Zookeeper, memory, and local files synchronize metadata for fault‑tolerant, high‑performance OLAP analytics.
ByteDance shares its real‑time data lake implementation, covering the evolving definition of data lakes, six core capabilities, challenges such as data management, weak concurrent updates, performance, and log ingestion, and detailed solutions including Hudi Metastore Server, bucket indexing, multi‑stage use cases, and future roadmap.
ByteDance’s Data Catalog article details the system’s unified metadata model, standardized ingestion connectors, search optimization techniques, lineage capabilities, and storage layer enhancements, highlighting key technical designs, performance improvements, and future work to advance data governance and asset utilization.
This article introduces ByteDance’s real‑time data lake platform built on Apache Hudi, covering Hudi fundamentals, table types, indexing, practical use cases, platform optimizations, and future roadmap, illustrating how the system enables low‑latency, scalable analytics across batch and streaming workloads.
This article outlines Ctrip Vacation's data standard management approach, covering why standards are needed, the three‑element framework of scope, tools, and policies, and detailed practices for data integration, production change handling, metadata governance, portal dashboard standardization, and self‑service query templating.
This article presents ByteDance’s ByteLake, a real‑time data lake platform built on Apache Hudi, covering Hudi fundamentals, ByteLake’s use cases, the platform’s architectural optimizations, new features such as a commit‑based metastore and bucket indexing, and future roadmap plans.
This article introduces OPPO's self‑developed data lake storage system CBFS, covering the fundamentals of data lake storage, the multi‑layer CBFS architecture, its core technologies such as metadata management and erasure coding, and future directions for large‑scale, low‑cost data analytics.
This article describes how Shuhai Supply Chain upgraded its data warehouse from a complex, high‑cost 1.0 architecture to a streamlined, real‑time solution built around Apache Doris, detailing the motivations, design choices, zero‑code ingestion, metadata management, Flink connector, and the resulting performance gains.