Skip to content
Tagged articles

data architecture

279 articles · Page 2 of 3
Architecture Practice Methods
Architecture Practice Methods
Feb 5, 2025 · Fundamentals

Data Architecture: Standard Definitions, Challenges, and Optimization Paths

This article defines data architecture per ISO/IEC/IEEE 42010 and DCMM, outlines core components — data model, flow, metadata, integration — identifies four key challenges including siloed management and technology-first approaches, and proposes optimization paths: enterprise-wide architectural governance, layered stable design, and collaborative data governance.

DCMMData Modelingdata architecture
0 likes · 8 min read
Data Architecture: Standard Definitions, Challenges, and Optimization Paths
Data Thinking Notes
Data Thinking Notes
Dec 29, 2024 · Information Security

A Complete Blueprint for Enterprise Digital Transformation Architecture

This article presents a comprehensive visual guide to enterprise digital transformation, covering overall digital planning, application system architecture, data architecture, information security architecture, and digital organization and governance, illustrating each layer with detailed diagrams to aid strategic implementation.

Governancedata architecturedigital transformation
0 likes · 5 min read
A Complete Blueprint for Enterprise Digital Transformation Architecture
DataFunSummit
DataFunSummit
Dec 29, 2024 · Big Data

Ant Group Data Architecture Practice and Upgrade Strategy

This article shares Ant Group's practical experience and evolution of data architecture, covering theoretical foundations, analysis of current challenges, proposed upgrade solutions based on domain‑driven design, and future outlook through complex network theory to improve scalability, governance, and resilience of large‑scale data systems.

Ant Groupdata architecturedomain-driven design
0 likes · 19 min read
Ant Group Data Architecture Practice and Upgrade Strategy
DataFunSummit
DataFunSummit
Dec 21, 2024 · Big Data

Big Data Implementation Practices and Architecture in a Foreign Bank

This article shares the foreign bank's big data implementation journey, covering background and goals, overall planning and architecture, practical insights, phased rollout, data governance, security, and Q&A, illustrating how a unified data platform, storage‑compute separation, and AI‑driven tools drive business innovation.

AIStorage-Compute Separationbanking
0 likes · 19 min read
Big Data Implementation Practices and Architecture in a Foreign Bank
58 Tech
58 Tech
Dec 19, 2024 · Big Data

Architecture Evolution and Implementation of the Intelligent Acceleration Engine in the 58 Big Data Platform

The article details the background, architectural analysis, multi‑tenant redesign, engine selection enhancements, compatibility adaptations, stability fixes, containerized deployment, performance optimizations, and measurable business outcomes of the Intelligent Acceleration Engine upgrade using Apache Kyuubi and StarRocks within the 58 big data platform.

Apache KyuubiContainerizationMulti-tenancy
0 likes · 12 min read
Architecture Evolution and Implementation of the Intelligent Acceleration Engine in the 58 Big Data Platform
AntData
AntData
Nov 18, 2024 · Databases

Modern Data Paradigms: From Relational Databases to Vector Retrieval and AI

This article surveys the evolution of modern data technologies—from the 4V characteristics of big data and the limitations of traditional relational databases, through the rise of NoSQL and polyglot persistence, to embedding‑driven vector search, hybrid retrieval and RAG, illustrating how each paradigm frees applications from data constraints.

EmbeddingNoSQLartificial intelligence
0 likes · 30 min read
Modern Data Paradigms: From Relational Databases to Vector Retrieval and AI
Bilibili Tech
Bilibili Tech
Oct 25, 2024 · Big Data

DataFunSummit2024: Next-Generation Data Architecture Technology Summit

DataFunSummit2024, co-hosted by Bilibili, convenes industry experts, scholars, and enterprise leaders across six forums to discuss next‑generation data architecture, showcasing Bilibili’s Iceberg‑based stream‑batch innovations, AI‑BI analytics, NoETL practices, and emerging alternatives to Lambda architecture.

AI+BIData EngineeringIceberg
0 likes · 3 min read
DataFunSummit2024: Next-Generation Data Architecture Technology Summit
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 22, 2024 · Big Data

Key Frameworks and Characteristics of Lakehouse Architecture: A Ground‑Level Perspective

This article reviews the emerging lakehouse architecture, outlines its core frameworks such as Hudi, Iceberg, Paimon, Flink, and Doris, discusses their storage‑compute separation, read‑write optimizations, and highlights how companies of different sizes adopt these technologies based on cost, efficiency, and specific business scenarios.

DorisFlinkOLAP
0 likes · 6 min read
Key Frameworks and Characteristics of Lakehouse Architecture: A Ground‑Level Perspective
DataFunTalk
DataFunTalk
Oct 3, 2024 · Big Data

Data Lake Technology Maturity Curve: Architecture, Design Principles, Core Functions, and Open‑Source Solutions

Amid growing data demands, this article explains the data lake technology maturity curve, detailing lake‑warehouse architectural patterns, design principles, core functionalities, and the four leading open‑source solutions (Hudi, Iceberg, Delta Lake, Paimon) to guide enterprises in building flexible, scalable, and governed data platforms.

Delta LakeHudiIceberg
0 likes · 10 min read
Data Lake Technology Maturity Curve: Architecture, Design Principles, Core Functions, and Open‑Source Solutions
JD Tech
JD Tech
Sep 28, 2024 · Big Data

From Early Coding to Big Data Architecture: A Personal Journey Through Data Platforms, Cloud Migration, and System Design

The article chronicles the author’s 30‑year programming career, detailing early experiences, the evolution from JavaScript projects to large‑scale big‑data architectures, cloud migration, business‑agnostic framework design, interactive analytics, and reflections on becoming an independent software architect.

big datacareer journeycloud computing
0 likes · 24 min read
From Early Coding to Big Data Architecture: A Personal Journey Through Data Platforms, Cloud Migration, and System Design
DataFunSummit
DataFunSummit
Sep 7, 2024 · Big Data

Observations on the Third Evolution of Data Infrastructure and the Next‑Generation Data Platform Architecture

This article reviews the current state of data platforms, analyzes the third wave of data infrastructure evolution driven by databases, big data and generative AI, proposes next‑generation lakehouse and cloud‑native architectural directions, and outlines future trends and unresolved challenges for AI‑centric data platforms.

Cloud Nativedata architecture
0 likes · 21 min read
Observations on the Third Evolution of Data Infrastructure and the Next‑Generation Data Platform Architecture
Data Thinking Notes
Data Thinking Notes
Jul 29, 2024 · Big Data

What Is a Data Middle Platform and How Does It Transform Enterprise Data Management?

This article explains the concept, design principles, and core components of a data middle platform, detailing its overall, functional, layered, logical, and data architectures, as well as the specific platforms for data collection, processing, organization, governance, quality, sharing, and visualization, illustrated with diagrams.

big datadata architecturedata governance
0 likes · 27 min read
What Is a Data Middle Platform and How Does It Transform Enterprise Data Management?
Data Thinking Notes
Data Thinking Notes
Jul 22, 2024 · Fundamentals

Why Data Architecture Governance Is the Key to Successful Digital Transformation

Data architecture governance, encompassing standards, security, modeling, quality, and lifecycle management, is essential for digital transformation in fast‑growing industries like express delivery, and this article outlines current challenges, traditional approaches, and a practical, phased methodology with platform support to implement effective governance.

Data Modelingdata architecturedata governance
0 likes · 12 min read
Why Data Architecture Governance Is the Key to Successful Digital Transformation
DataFunTalk
DataFunTalk
Jul 17, 2024 · Databases

From DIKW to Distributed Data Warebase: Letting Data Emerge as Intelligence

This article explores the DIKW hierarchy, explains how data evolves into information, knowledge, and wisdom, examines traditional data models and products, critiques existing multi‑system architectures, and proposes a new distributed Data Warebase that unifies structured, semi‑structured, and vectorized knowledge to enable intelligent data-driven applications.

DIKWData SystemsDistributed Data Warehouse
0 likes · 24 min read
From DIKW to Distributed Data Warebase: Letting Data Emerge as Intelligence
DataFunSummit
DataFunSummit
Jul 2, 2024 · Cloud Computing

Global Perspective on Multi-Cloud Data Architecture

The forum presents a series of technical talks on multi‑cloud data architecture, covering Xiaomi’s lake‑warehouse practice, cross‑border e‑commerce data platforms, Alluxio‑based machine‑learning acceleration, Qichacha’s cost‑effective data solutions, and Kuaishou’s Flink on Kubernetes migration, highlighting strategies, implementations, and audience benefits.

Multi-cloudbig datacloud computing
0 likes · 8 min read
Global Perspective on Multi-Cloud Data Architecture
DevOps
DevOps
Jun 20, 2024 · Fundamentals

Understanding the 4A Enterprise Architecture: Business, Technology, Application, and Data Architecture

This article explains the TOGAF framework and its 4A architecture model—Business, Technology, Application, and Data Architecture—detailing each domain's definition, purpose, value, and step‑by‑step guidance for creating enterprise architecture diagrams to align strategy, processes, and technology.

Application ArchitectureTOGAFTechnology Architecture
0 likes · 8 min read
Understanding the 4A Enterprise Architecture: Business, Technology, Application, and Data Architecture
Architecture Practice Methods
Architecture Practice Methods
Jun 19, 2024 · R&D Management

Huawei's Business Object Framework: Principles, Architecture Placement, and TOGAF Comparison

This article explains Huawei's definition of business objects as key people, things, and events that unify business concepts and bridge business-IT communication, detailing their dual nature as physical/logical entities and information carriers, four abstraction principles, and their placement in data architecture within Huawei's V-model versus TOGAF 10's business information approach.

HuaweiTOGAFV-model
0 likes · 6 min read
Huawei's Business Object Framework: Principles, Architecture Placement, and TOGAF Comparison
Code Ape Tech Column
Code Ape Tech Column
May 28, 2024 · Databases

Query Separation: A Practical Approach to Optimizing Large Table Reads

This article explains the concept of query separation, outlines its suitable scenarios, compares implementation methods such as synchronous, asynchronous, and binlog approaches, discusses storage system choices like MongoDB, HBase, and Elasticsearch, and addresses consistency and operational challenges when decoupling read workloads from write workloads.

ElasticsearchMQdata architecture
0 likes · 8 min read
Query Separation: A Practical Approach to Optimizing Large Table Reads
DataFunTalk
DataFunTalk
May 1, 2024 · Artificial Intelligence

A Comprehensive Guide to Vector Database Architecture and Application Scenarios

This article provides a detailed overview of vector database structures, their evolution, enterprise challenges, functional features, future trends, and key use cases, illustrating how they serve as the memory engine for large AI models and support multimodal data processing in modern data architectures.

AIdata architectureenterprise
0 likes · 11 min read
A Comprehensive Guide to Vector Database Architecture and Application Scenarios
DataFunTalk
DataFunTalk
Mar 29, 2024 · Big Data

Best Practices for Building an International Ride‑Hailing Data Metric System at Didi

This article outlines Didi’s best‑practice approach to constructing a global ride‑hailing data metric system, covering business scenarios, metric categories, pain points such as definition and technical challenges, and a comprehensive solution involving organizational structure, processes, model design, tooling, timezone handling, and governance.

DidiMulti‑Timezonedata architecture
0 likes · 13 min read
Best Practices for Building an International Ride‑Hailing Data Metric System at Didi
Sohu Tech Products
Sohu Tech Products
Mar 13, 2024 · Databases

DingoDB Multi-Modal Vector Database: Design Philosophy, Architecture and Applications

DingoDB is a multi‑modal vector database that unifies storage and analysis of structured, semi‑structured and unstructured data through a Raft‑based distributed architecture, offering MySQL‑compatible SQL, high‑performance APIs, automatic sharding, real‑time index optimization, and hybrid scalar‑vector queries for enterprise knowledge bases, LLM memory, and real‑time decision‑making.

DingoDBLLM ApplicationsMulti-Modal Database
0 likes · 11 min read
DingoDB Multi-Modal Vector Database: Design Philosophy, Architecture and Applications
DataFunSummit
DataFunSummit
Feb 19, 2024 · Big Data

Yipay Data Warehouse Construction and Data Governance Practices

This presentation by senior data warehouse engineer Huang Luo details Yipay's end‑to‑end data warehouse build, covering background challenges, governance framework, platform development, layered architecture, naming standards, monitoring, and future plans, offering practical insights for data engineers, architects, and business stakeholders.

big datadata architecturedata quality
0 likes · 14 min read
Yipay Data Warehouse Construction and Data Governance Practices
vivo Internet Technology
vivo Internet Technology
Jan 24, 2024 · Big Data

Evolution of Vivo's Trillions-Scale Data Architecture: Dual-Active Real-Time and Offline Computing

Vivo’s trillion‑scale data platform evolved into a dual‑active real‑time and offline architecture that leverages multi‑datacenter clusters, Kafka/Pulsar caching, a unified sorting layer, HBase‑backed dimension tables, and micro‑batch Spark jobs to deliver low‑cost, high‑performance processing, 99.9% availability, and 99.9995% data‑integrity.

HBaseOffline Computingdata architecture
0 likes · 16 min read
Evolution of Vivo's Trillions-Scale Data Architecture: Dual-Active Real-Time and Offline Computing
Architects Research Society
Architects Research Society
Jan 15, 2024 · Fundamentals

Evolution of Software Architecture Styles and Domains

This article outlines the evolution of software architecture styles, describing various architectural domains and sub‑domains—from web and mobile applications to integration, data, and analytics architectures—and their typical implementations, illustrated with a detailed classification table.

AnalyticsDomain Architecturearchitecture styles
0 likes · 8 min read
Evolution of Software Architecture Styles and Domains
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Jan 10, 2024 · Big Data

CaoCao Mobility's Real‑Time Data Warehouse: Hologres + Flink

This article details how CaoCao Mobility transformed its ride‑hailing platform by replacing a traditional Lambda architecture with an enterprise‑grade real‑time data warehouse built on Hologres and Flink, covering business motivations, architectural design, component capabilities, performance optimizations, operational safeguards, and future roadmap.

FlinkHologresdata architecture
0 likes · 19 min read
CaoCao Mobility's Real‑Time Data Warehouse: Hologres + Flink
Data Thinking Notes
Data Thinking Notes
Jan 9, 2024 · Fundamentals

How to Build an Effective Enterprise Data Governance Framework

Designing a data governance system involves defining data architecture, standards, and master data, documenting mechanisms, aligning them with actual business and analytical data processes, forming joint teams for specialized governance activities, and institutionalizing these practices through formal processes to sustain continuous data capability across the enterprise.

Enterprise DataMaster Datadata architecture
0 likes · 3 min read
How to Build an Effective Enterprise Data Governance Framework
Architects Research Society
Architects Research Society
Jan 2, 2024 · Big Data

Understanding Data Lakes: Concepts, Benefits, Challenges, and Comparison with Data Warehouses

This article explains what a data lake is, its origins, key characteristics such as collecting all data, enabling diverse user access, and flexible processing, compares it with traditional data warehouses, discusses cost advantages, potential pitfalls like data swamps, and outlines best‑practice considerations for enterprise adoption.

AnalyticsHadoopdata architecture
0 likes · 10 min read
Understanding Data Lakes: Concepts, Benefits, Challenges, and Comparison with Data Warehouses
DataFunTalk
DataFunTalk
Dec 18, 2023 · Big Data

Unified Data Architecture: Balancing Freshness, Cost, and Performance with Incremental Computing

The article explains why unified data architecture is essential to avoid duplication and inefficiency, discusses differing performance trade‑offs among batch, streaming, and interactive analytics, introduces an incremental computation model that unifies these modes, and invites readers to a Dec 19, 2023 technical sharing event.

Lambda architecturebatch processingbig data
0 likes · 3 min read
Unified Data Architecture: Balancing Freshness, Cost, and Performance with Incremental Computing
Architects Research Society
Architects Research Society
Nov 26, 2023 · Big Data

Data Lake vs Data Warehouse: Key Differences and How to Choose

Data lakes and data warehouses serve different purposes in big‑data architectures; this article explains their definitions, core attributes, five major distinctions—including data retention, type support, user coverage, adaptability, and insight speed—and offers guidance on selecting or combining the two approaches.

AnalyticsHadoopdata architecture
0 likes · 12 min read
Data Lake vs Data Warehouse: Key Differences and How to Choose
DataFunSummit
DataFunSummit
Oct 22, 2023 · Big Data

How Kuaishou E‑commerce Leverages OLAP and a Unified Data Architecture to Solve Business Data Challenges

This article explains how Kuaishou's e‑commerce team built a unified OLAP‑based data platform—covering data ingestion, consistent dimensional and fact layers, metric management, and real‑time services—to address rapid growth, metric inconsistency, and operational inefficiencies across multiple business scenarios.

E‑commerceMetricsOLAP
0 likes · 20 min read
How Kuaishou E‑commerce Leverages OLAP and a Unified Data Architecture to Solve Business Data Challenges
DataFunSummit
DataFunSummit
Oct 18, 2023 · Big Data

Kuaishou Data Lake Construction with Apache Hudi: Architecture, Challenges, and Solutions

This article explains why Kuaishou built a data lake, outlines the shortcomings of its previous Lambda architecture, describes the adoption of Apache Hudi for unified batch‑stream processing, and details the five major technical challenges and the corresponding solutions implemented to improve performance, consistency, and operational reliability.

Apache HudiFlinkbig data
0 likes · 17 min read
Kuaishou Data Lake Construction with Apache Hudi: Architecture, Challenges, and Solutions
Smart Era Software Development
Smart Era Software Development
Aug 25, 2023 · Big Data

Evolving Enterprise Data Architecture for the Large‑Model Era: Practices and Case Studies

The article analyzes how enterprise data systems must be re‑engineered for large‑model applications, outlines the three‑stage data pipeline (ingestion, orchestration, interaction), introduces data‑virtualization techniques with virtual tables and intelligent materialization, and validates the approach with two banking case studies.

Data PipelineVector Databasebig data
0 likes · 14 min read
Evolving Enterprise Data Architecture for the Large‑Model Era: Practices and Case Studies
DataFunSummit
DataFunSummit
Aug 5, 2023 · Big Data

Manbang Group's Real-Time Computing, Data Architecture, and Product Practices

Manbang Group shares its practical experiences and insights on real-time computing, multi‑cloud platform architecture, data warehousing with Flink and Holo, real‑time decision and feature platforms, and future plans for scaling these systems to support logistics and recommendation algorithms.

Cloud NativeFlinkHolo
0 likes · 16 min read
Manbang Group's Real-Time Computing, Data Architecture, and Product Practices
Alibaba Cloud Developer
Alibaba Cloud Developer
Jul 31, 2023 · Big Data

From BI to Kappa: How Data Architecture Evolved in the Big Data Era

This article traces the evolution of data architecture from early BI systems through traditional big‑data stacks, streaming, Lambda and Kappa designs, and explains how a unified stream‑batch model simplifies development while keeping logic consistent across data‑analysis and pipeline applications.

BI systemsKappa architectureLambda architecture
0 likes · 16 min read
From BI to Kappa: How Data Architecture Evolved in the Big Data Era
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 8, 2023 · Big Data

The Pros and Cons of the Middle‑Platform Model in Large Enterprises: A Data Department Perspective

This article examines the middle‑platform concept in large companies, especially data departments, outlining its benefits such as resource centralisation, reduced duplication and experience sharing, while also highlighting drawbacks like slow response, excessive authority, and the need to bring platform capabilities closer to business units.

data architectureenterprisemiddle platform
0 likes · 7 min read
The Pros and Cons of the Middle‑Platform Model in Large Enterprises: A Data Department Perspective
Big Data Technology Architecture
Big Data Technology Architecture
Jun 20, 2023 · Big Data

Data Platform Evolution and the Future of Snowflake in China: Insights from Industry Leaders

The panel discusses the three‑stage evolution of data platforms, compares US and Chinese market dynamics, evaluates Snowflake’s success factors, and outlines the criteria and opportunities for a China‑specific Snowflake‑like solution, while also sharing investment perspectives on data‑driven startups.

Data PlatformsSnowflakecloud
0 likes · 24 min read
Data Platform Evolution and the Future of Snowflake in China: Insights from Industry Leaders
Data Thinking Notes
Data Thinking Notes
Jun 18, 2023 · Big Data

Data Lake vs Data Warehouse: Uncover the Real Differences

This article explores the evolving concept of data lakes, compares them with traditional data warehouses across storage, modeling, tooling, and user roles, and examines the emerging lake‑warehouse integration, highlighting why both remain essential in modern big‑data architectures.

big datadata architecturedata lake
0 likes · 12 min read
Data Lake vs Data Warehouse: Uncover the Real Differences
Top Architect
Top Architect
May 4, 2023 · Big Data

Data Middle Platform: General Architecture and Core Components

The article explains the concept, benefits, and detailed modular architecture of a data middle platform, covering data storage, acquisition, processing, governance, security, and operation frameworks, and illustrates how enterprises can build and evolve such platforms to turn data into valuable services.

big datadata architecturedata governance
0 likes · 19 min read
Data Middle Platform: General Architecture and Core Components
DataFunTalk
DataFunTalk
Apr 21, 2023 · Fundamentals

Data Architecture and Data Modeling Overview, Solutions, and Enterprise Case Studies

This article explains data architecture and data modeling fundamentals, presents DAMA DMBOK concepts, outlines four practical solutions for model design, standard management, automated change control, and business mapping, and shares an enterprise manufacturing case study with Q&A on governance and efficiency.

Data ModelingEnterprise Datadata architecture
0 likes · 21 min read
Data Architecture and Data Modeling Overview, Solutions, and Enterprise Case Studies
DataFunTalk
DataFunTalk
Apr 6, 2023 · Product Management

Designing and Implementing a Channel Data Product for Growth

This article explains why channel data products are needed, outlines the stages of growth business, describes how to design and implement a channel data product—including its architecture, database schema, and operational workflow—and concludes with a practical summary and Q&A.

Business Intelligencechannel datadata analytics
0 likes · 11 min read
Designing and Implementing a Channel Data Product for Growth
DataFunTalk
DataFunTalk
Mar 15, 2023 · Big Data

Evolution of Next‑Generation Cloud Data Platform Architecture

This technical presentation reviews the historical development of big data platforms, outlines the four generations of cloud data platform architectures, details the modern cloud‑native stack—including unified metadata, scheduling, and integration systems—and showcases a real‑world industrial manufacturing case with a Q&A session.

Cloud Data PlatformSchedulingdata architecture
0 likes · 23 min read
Evolution of Next‑Generation Cloud Data Platform Architecture
DataFunSummit
DataFunSummit
Mar 9, 2023 · Big Data

Designing Efficient and Agile Real-Time Big Data Analytics Platforms for Enterprises

The article explains how enterprises can build a comprehensive big data analytics platform—covering data collection, storage, computation, and decision layers—by clarifying business scenarios, choosing appropriate on‑premise or cloud deployment, selecting suitable architectures such as Lambda/Kappa, and addressing component choices and emerging technical trends.

analytics platformbig datacloud deployment
0 likes · 9 min read
Designing Efficient and Agile Real-Time Big Data Analytics Platforms for Enterprises
macrozheng
macrozheng
Feb 28, 2023 · Big Data

How Tencent Music Scaled Its Content Data Platform with Apache Doris: From ClickHouse to 4.0 Architecture

This article details the evolution of Tencent Music's content data platform from version 1.0 to 4.0, describing the migration from ClickHouse to Apache Doris, the introduction of a semantic layer, optimization of data ingestion, query performance, and cost reduction strategies that dramatically improved data timeliness, operational efficiency, and storage costs.

Apache DorisFlinkOptimization
0 likes · 23 min read
How Tencent Music Scaled Its Content Data Platform with Apache Doris: From ClickHouse to 4.0 Architecture
DataFunSummit
DataFunSummit
Feb 13, 2023 · Big Data

ClickHouse in Self‑Service Analytics: Architecture, Optimization Practices and Future Roadmap at ZuanZuan Platform

This article details how ZuanZuan leveraged ClickHouse as the core OLAP engine for its massive self‑service analytics platform, covering OLAP engine selection criteria, system architecture, real‑world use cases, performance tuning, operational challenges, and future development plans.

AnalyticsClickHouseDistributed Systems
0 likes · 16 min read
ClickHouse in Self‑Service Analytics: Architecture, Optimization Practices and Future Roadmap at ZuanZuan Platform
iQIYI Technical Product Team
iQIYI Technical Product Team
Feb 3, 2023 · Big Data

Data Lake Concepts, Benefits, and Iceberg‑Based Implementations at iQIYI

iQIYI’s data lake combines public‑cloud and private storage with Apache Iceberg’s snapshot‑based table format to enable near‑real‑time, unified batch‑and‑stream analytics, reducing costs, simplifying architecture, and improving data freshness across use cases such as log collection, audit, pingback, and member order processing.

Apache IcebergStreaming‑Batch Integrationdata architecture
0 likes · 25 min read
Data Lake Concepts, Benefits, and Iceberg‑Based Implementations at iQIYI
dbaplus Community
dbaplus Community
Dec 13, 2022 · Big Data

How ClickHouse Powers Real-Time Self-Service Analytics at Scale

Facing massive daily data volumes and complex, ad‑hoc analytical needs, Zhaozhuan’s engineering team evaluated multiple OLAP engines and chose ClickHouse, then built a four‑layer self‑service analytics platform, detailing architecture, use‑cases, performance tuning, large‑scale joins, and future roadmap challenges.

ClickHouseOLAPPerformance Tuning
0 likes · 14 min read
How ClickHouse Powers Real-Time Self-Service Analytics at Scale
ITPUB
ITPUB
Dec 10, 2022 · Big Data

How ClickHouse Powers Real-Time Self-Service Analytics at Scale

This article examines why ClickHouse was chosen as the OLAP engine for a massive self‑service analytics platform, describes the system architecture, shares concrete memory and performance tuning parameters, and outlines current challenges and future roadmap for large‑scale real‑time data analysis.

ClickHouseOLAPPerformance Tuning
0 likes · 14 min read
How ClickHouse Powers Real-Time Self-Service Analytics at Scale
Zhuanzhuan Tech
Zhuanzhuan Tech
Dec 7, 2022 · Databases

ClickHouse in Self‑Service Analytics: OLAP Selection, Platform Architecture, Optimization Practices, and Future Outlook

This article examines the selection of ClickHouse as the OLAP engine for a self‑service analytics platform, describes the platform’s architecture, details memory and performance tuning techniques, discusses large‑scale join handling, and outlines current challenges and future development directions for ClickHouse.

ClickHouseOLAPPerformance Optimization
0 likes · 12 min read
ClickHouse in Self‑Service Analytics: OLAP Selection, Platform Architecture, Optimization Practices, and Future Outlook
Architects Research Society
Architects Research Society
Nov 18, 2022 · Databases

Formal Naming of Data Schemas, Structures, and Models: Distinctions and Hierarchies

The article explains how to formally name data schemas, structures, and models, clarifies the differences between internal, external, logical, and physical schemas, and proposes a systematic hierarchy—including strategic and tactical schemas—to improve data architecture and reduce confusion in enterprise data design.

Information SystemsSchema Namingdata architecture
0 likes · 11 min read
Formal Naming of Data Schemas, Structures, and Models: Distinctions and Hierarchies
DevOps Cloud Academy
DevOps Cloud Academy
Nov 5, 2022 · Fundamentals

Understanding Data Architecture: Definitions, Problems Solved, Core Components, and Future Trends

This article explains what data architecture is, why it is essential for linking business and technology, outlines its main components such as data models, data flows, value streams and standards, and discusses emerging trends toward service‑oriented, consumption‑focused data architectures.

Data ManagementData Modelingdata architecture
0 likes · 9 min read
Understanding Data Architecture: Definitions, Problems Solved, Core Components, and Future Trends
DataFunTalk
DataFunTalk
Oct 17, 2022 · Big Data

How Data Empowers the Fast‑Moving Consumer Goods Industry: Baicaowei’s End‑to‑End Data Platform Evolution

This article details Baicaowei’s journey from a Hadoop‑based data platform to a modern StarRocks‑driven architecture, illustrating how digitalization, evolving business needs, and streamlined data pipelines empower the fast‑moving consumer goods sector through efficient data collection, modeling, and analytics.

StarRocksbig dataconsumer goods
0 likes · 10 min read
How Data Empowers the Fast‑Moving Consumer Goods Industry: Baicaowei’s End‑to‑End Data Platform Evolution
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Sep 13, 2022 · Big Data

From Hadoop to Cloud‑Native: The Evolution of Data Lakes and Modern Architecture

This article traces the history of data lakes from their 2010 inception with Hadoop through cloud‑native object storage, lakehouse formats like Delta Lake, and Alibaba Cloud's multi‑layer solution, outlining key architectural stages and practical construction challenges for enterprise‑grade implementations.

Alibaba CloudCloud Nativebig data
0 likes · 9 min read
From Hadoop to Cloud‑Native: The Evolution of Data Lakes and Modern Architecture
Efficient Ops
Efficient Ops
Aug 14, 2022 · Databases

How TDengine 3.0 Redefines Cloud‑Native Time‑Series Databases

The TDengine Developer Conference in Beijing unveiled the open‑source, cloud‑native TDengine 3.0, detailing its revolutionary architecture that tackles high‑cardinality challenges, introduces RAFT‑based distribution, and showcases real‑world IoT and IT‑operations case studies where enterprises dramatically improved performance and reduced costs.

IoTOpen SourceTDengine
0 likes · 11 min read
How TDengine 3.0 Redefines Cloud‑Native Time‑Series Databases
GuanYuan Data Tech Team
GuanYuan Data Tech Team
Aug 4, 2022 · Cloud Native

What Is a Cloud‑Native Data Platform? Architecture, Components, and Best Practices

This article explores the evolution and architecture of cloud‑native data platforms, covering their historical roots, modern components such as storage layers, ingestion, processing, metadata, and consumption, and offers practical guidance on selecting tools, designing pipelines, and implementing best‑practice strategies for scalable, flexible data infrastructure.

big-datacloud-nativedata architecture
0 likes · 41 min read
What Is a Cloud‑Native Data Platform? Architecture, Components, and Best Practices
Past Memory Big Data
Past Memory Big Data
Jul 22, 2022 · Big Data

Choosing Modern Data Architecture: Data Fabric vs. Data Mesh

The article compares Data Fabric and Data Mesh as modern data‑architecture approaches, explains their technical and organizational differences, discusses the ongoing debate between data lakes, warehouses, and lakehouses, and highlights how each option fits varying data‑type and usage scenarios.

Data Fabricdata architecturedata lake
0 likes · 4 min read
Choosing Modern Data Architecture: Data Fabric vs. Data Mesh
High Availability Architecture
High Availability Architecture
Jul 7, 2022 · Big Data

Interview with Tencent Cloud’s Zhang Zhigang on Lakehouse Architecture and Cloud‑Native Integration

In this interview, Tencent Cloud expert Zhang Zhigang explains the fundamentals and key technologies of lakehouse architecture, discusses how cloud‑native practices enhance its performance and operability, and offers practical advice for big‑data professionals ahead of the 2022 GIAC Global Internet Architecture Conference in Shenzhen.

Cloud NativeTencent Clouddata architecture
0 likes · 10 min read
Interview with Tencent Cloud’s Zhang Zhigang on Lakehouse Architecture and Cloud‑Native Integration
Baidu Geek Talk
Baidu Geek Talk
Jul 1, 2022 · Big Data

Evolution of Data Platform Technology: From Data Warehouse to Lakehouse Architecture

The article traces the evolution of data platforms from early data warehouses—using schema‑on‑write, columnar storage, and MPP engines—to data lakes that retain raw data with schema‑on‑read, and finally to lakehouse architectures that merge storage and compute, offering unified metadata, versioning, and support for BI, big‑data, AI, and HPC workloads.

OLAPStorage-Compute Separationbig data platform
0 likes · 25 min read
Evolution of Data Platform Technology: From Data Warehouse to Lakehouse Architecture
Architects Research Society
Architects Research Society
Jun 25, 2022 · Fundamentals

Data Architecture: Definition, Goals, Principles, Components, and Best Practices

This article explains data architecture as the transformation of business needs into data and system requirements, outlines its objectives, core principles, essential components, the relationship with data modeling, relevant frameworks, and modern best‑practice guidelines for building scalable, cloud‑native, AI‑enabled architectures.

AI IntegrationCloud Nativedata architecture
0 likes · 10 min read
Data Architecture: Definition, Goals, Principles, Components, and Best Practices
Architecture Digest
Architecture Digest
May 16, 2022 · Databases

Evolution of JD Baitiao’s Data Architecture: From MySQL to Apache ShardingSphere

This article chronicles JD Baitiao’s journey from a monolithic MySQL setup through NoSQL and DBRep to a mature ShardingSphere‑based sharding solution, highlighting the technical motivations, architectural decoupling strategies, evaluation criteria, performance comparison, and the operational benefits achieved for a high‑traffic financial service.

Apache ShardingSpheredata architecturedatabases
0 likes · 14 min read
Evolution of JD Baitiao’s Data Architecture: From MySQL to Apache ShardingSphere
DataFunSummit
DataFunSummit
Apr 9, 2022 · Big Data

Impala Deployment and Optimization: Practical Experience with Sensor Data Multi‑dimensional Analysis Platform

This article presents a comprehensive technical walkthrough of Sensor Data's multi‑dimensional analysis platform, covering product architecture, an Impala‑based real‑time query engine, query performance tuning, resource‑estimation strategies, and future plans, with concrete diagrams, test results, and community contributions.

ImpalaQuery Optimizationbig data
0 likes · 19 min read
Impala Deployment and Optimization: Practical Experience with Sensor Data Multi‑dimensional Analysis Platform
dbaplus Community
dbaplus Community
Apr 1, 2022 · Databases

How iQIYI Built a Scalable OLTP Data Center to Eliminate Data Silos

This article details iQIYI's design and implementation of a unified OLTP data center that consolidates data across business lines, solves data‑island issues, ensures strong consistency between MongoDB and Elasticsearch, and provides high‑availability, massive‑scale storage for billions of records.

ElasticsearchHigh AvailabilityMongoDB
0 likes · 12 min read
How iQIYI Built a Scalable OLTP Data Center to Eliminate Data Silos
DataFunTalk
DataFunTalk
Feb 19, 2022 · Big Data

Fundamentals of Data Middle Platform: Logic, Principles, and Practice

This article explains what a data middle platform is, why organizations need it, its core principles, technical architecture, and practical implementation guidelines, highlighting how it solves issues like inconsistent metrics, duplicate data construction, low query efficiency, poor data quality, and high development costs.

big datadata architecturedata middle platform
0 likes · 14 min read
Fundamentals of Data Middle Platform: Logic, Principles, and Practice
DataFunSummit
DataFunSummit
Jan 23, 2022 · Big Data

MobTech's Integrated Data Governance Practices and Architecture

This article presents MobTech's comprehensive data governance and security practices, covering the necessity of governance, challenges in large‑scale data environments, the full‑link governance chain, modular architecture, and specific implementations for financial risk‑control scenarios.

Data Managementbig datadata architecture
0 likes · 19 min read
MobTech's Integrated Data Governance Practices and Architecture
Architecture Digest
Architecture Digest
Jan 21, 2022 · Big Data

Building a Real-Time Data Warehouse with Flink: Architecture, Core Concepts, and Practical Implementation

This article explains how to build a unified stream‑batch real‑time data warehouse using FlinkSQL, covering prerequisite knowledge, five core concepts, two implementation approaches, a comparison of traditional versus real‑time architectures, and a comprehensive hands‑on example, illustrated with diagrams.

Flinkbatch processingdata architecture
0 likes · 6 min read
Building a Real-Time Data Warehouse with Flink: Architecture, Core Concepts, and Practical Implementation
Architects' Tech Alliance
Architects' Tech Alliance
Dec 26, 2021 · Big Data

Understanding Data Lakes: Concepts, Architecture, Vendor Solutions, and Implementation Practices

This comprehensive article explains what a data lake is, its core characteristics, reference architecture, major cloud vendor implementations, typical use cases such as advertising and gaming, step‑by‑step construction guidance, and future trends in cloud‑native big‑data platforms.

Data ManagementVendor Solutionsdata architecture
0 likes · 51 min read
Understanding Data Lakes: Concepts, Architecture, Vendor Solutions, and Implementation Practices
High Availability Architecture
High Availability Architecture
Dec 23, 2021 · Fundamentals

Master Data Management Architecture and Practices for Baidu Smart Mini Programs

This article presents a comprehensive overview of master data management concepts, maturity levels, and the challenges faced by Baidu smart mini‑programs, followed by a detailed practical architecture design—including domain modeling, high‑availability microservice implementation, performance optimization, and data synchronization—while also discussing future extensions and team capability building.

Baidu Mini ProgramsHigh Availabilitydata architecture
0 likes · 14 min read
Master Data Management Architecture and Practices for Baidu Smart Mini Programs
21CTO
21CTO
Nov 8, 2021 · Big Data

How Baidu iFanFan Built a Real-Time Big Data Platform: Challenges & Lessons

Facing rapid business iteration, Baidu’s iFanFan data team designed a unified real‑time and offline big‑data platform, tackling business, technical, and organizational challenges through Lambda/Kappa architectures, data integration, storage, computation, governance, and scalable analytics to deliver timely, accurate, and valuable data products.

big datadata architecturedata warehouse
0 likes · 33 min read
How Baidu iFanFan Built a Real-Time Big Data Platform: Challenges & Lessons
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 14, 2021 · Big Data

Overview of Big Data Architecture Trends and Curated Resources

This article, discovered on the Yunqi community site, provides a system‑architecture perspective overview of current big‑data architecture hotspots, development trajectories, emerging trends, and unresolved challenges, while highlighting the field’s rapid evolution and recommending a curated list of in‑depth resources for further study.

Data EngineeringResourcesdata architecture
0 likes · 5 min read
Overview of Big Data Architecture Trends and Curated Resources
Java High-Performance Architecture
Java High-Performance Architecture
Oct 12, 2021 · Big Data

Unpacking the Core Technologies Behind Modern Big Data Platforms

This article breaks down a typical big data platform architecture into its four layers—data collection, storage and analysis, sharing, and real‑time computation—detailing the essential tools such as Flume, HDFS, Hive, Spark, DataX, and task scheduling systems that enable scalable, low‑latency data processing and delivery.

DataXHadoopSpark
0 likes · 8 min read
Unpacking the Core Technologies Behind Modern Big Data Platforms
Architecture Digest
Architecture Digest
Oct 11, 2021 · Big Data

Core Technologies and Architecture of a Big Data Platform

This article explains the typical architecture of a big‑data platform, detailing its four core layers—data collection, storage & analysis, data sharing, and application—and describing the key technologies such as Flume, DataX, HDFS, Hive, Spark, Spark Streaming, and task scheduling components.

Data IngestionDataXFlume
0 likes · 8 min read
Core Technologies and Architecture of a Big Data Platform
Architects' Tech Alliance
Architects' Tech Alliance
Sep 2, 2021 · Big Data

Core Technologies and Architecture of a Big Data Platform

The article outlines a typical big data platform architecture, detailing its core layers—data collection, storage and analysis, sharing, application, real-time computation, and task scheduling—while describing key technologies such as Flume, DataX, HDFS, Hive, Spark, Spark Streaming, and Redis.

HadoopSparkdata architecture
0 likes · 9 min read
Core Technologies and Architecture of a Big Data Platform
Alibaba Cloud Developer
Alibaba Cloud Developer
Sep 2, 2021 · Databases

From LAMP to Cloud‑Native: Evolving Application Data Architecture and Best Practices

This article traces two decades of application data architecture evolution, comparing traditional single‑system LAMP designs with modern multi‑component cloud‑native stacks, and offers practical guidance on scaling, component selection, CDC‑based data derivation, and cloud‑native implementations such as Tablestore.

CDCdata architecturedatabases
0 likes · 22 min read
From LAMP to Cloud‑Native: Evolving Application Data Architecture and Best Practices
IT Architects Alliance
IT Architects Alliance
Sep 1, 2021 · Big Data

Understanding Data Middle Platform Architecture and Its Core Components

The article explains the concept of a data middle platform, describing its architecture, the essential big‑data foundation, metadata management, data service components such as BI and tag systems, and how these layers together enable unified data access, governance, and business intelligence across enterprises.

Business IntelligenceTag Managementdata architecture
0 likes · 14 min read
Understanding Data Middle Platform Architecture and Its Core Components
Efficient Ops
Efficient Ops
Aug 31, 2021 · Cloud Computing

Why Object Storage Is the New Backbone of Cloud Data Management

This article explains how object storage emerged as a cloud-native solution that surpasses traditional DAS, SAN, and NAS architectures by offering virtually unlimited capacity, robust metadata handling, and simple RESTful APIs for modern applications and large‑scale data workloads.

Object Storagecloud storagedata architecture
0 likes · 11 min read
Why Object Storage Is the New Backbone of Cloud Data Management
JD Retail Technology
JD Retail Technology
Aug 12, 2021 · Big Data

Design and Implementation of JD Mini‑Program Custom Data Analysis Service

This article presents the technical solution and key processes of JD's mini‑program custom data analysis service, covering business background, ClickHouse‑based storage design, real‑time processing pipelines, dynamic rule parsing, table architecture, monitoring mechanisms, and future outlook for large‑scale data analytics.

ClickHouseCustom Data AnalysisJD Mini-Program
0 likes · 13 min read
Design and Implementation of JD Mini‑Program Custom Data Analysis Service
Qingyun Technology Community
Qingyun Technology Community
Aug 3, 2021 · Cloud Computing

How QingStor’s Object Storage Architecture Powers Massive Data Scalability

This article explains QingStor's object storage concepts, core advantages, global data model, subsystem design, massive small‑file optimizations, key features like lifecycle management and cross‑region replication, and showcases a traffic‑industry use case, highlighting its scalability, reliability, and ease of integration.

Object StorageQingStorcloud storage
0 likes · 20 min read
How QingStor’s Object Storage Architecture Powers Massive Data Scalability
Airbnb Technology Team
Airbnb Technology Team
Jul 29, 2021 · Big Data

Airbnb’s Data Quality Improvement Plan: Organizational, Architectural, and Governance Practices

Airbnb’s 2019 Data Quality Improvement Plan reorganized its data‑engineering workforce, introduced a dedicated data‑engineer role, adopted a decentralized Minerva‑based architecture with Spark pipelines, instituted rigorous testing, governance, and certification processes, and established SLAs and monitoring to ensure timely, trustworthy, well‑documented data across the enterprise.

AirbnbData Engineeringbig data
0 likes · 13 min read
Airbnb’s Data Quality Improvement Plan: Organizational, Architectural, and Governance Practices
IT Architects Alliance
IT Architects Alliance
Jul 20, 2021 · Big Data

Understanding Data Middle Platform: Layers, Architecture, and Implementation Methodology

The article explains the concept of a data middle platform, detailing its three-layer structure—data model, data service, and data development—illustrates how data modeling enables cross-domain integration, how services encapsulate data for flexible consumption, and how development tools support customized data applications, using a telecom operator example.

Data Modelingbig datadata architecture
0 likes · 2 min read
Understanding Data Middle Platform: Layers, Architecture, and Implementation Methodology
dbaplus Community
dbaplus Community
Jun 2, 2021 · Databases

How to Build a Mature Data Warehouse: 7 Essential Steps and Best Practices

This article explains why data warehouses are critical for decision‑making, outlines the challenges of immature warehouses, and provides a step‑by‑step framework—including goal setting, technology selection, problem identification, domain modeling, layer design, modeling principles, and governance standards—to help teams build a robust, maintainable data warehouse.

ETLbig datadata architecture
0 likes · 22 min read
How to Build a Mature Data Warehouse: 7 Essential Steps and Best Practices
Big Data Technology & Architecture
Big Data Technology & Architecture
May 27, 2021 · Databases

Database Selection and TiDB Implementation in NetEase Interactive Entertainment Billing Group

This article details the billing group's challenges with single‑node MySQL, the evaluation of alternative databases such as TiDB and CockroachDB, performance testing, migration strategies, operational best practices, and the final decision to adopt TiDB for scalable, high‑availability data services.

Cloud NativeDatabase MigrationTiDB
0 likes · 15 min read
Database Selection and TiDB Implementation in NetEase Interactive Entertainment Billing Group
Architects Research Society
Architects Research Society
May 23, 2021 · Big Data

Data Architecture Trends: From Chaos to an Organized Era – Insights from Anthony J. Algmin

The article reviews Anthony J. Algmin’s reflections on past data‑architecture predictions, current hot topics such as cloud, AI/ML, data governance, and real‑time analytics, and forecasts future trends including metadata management, blockchain, and the evolving role of data architects within enterprises.

artificial intelligencebig datadata architecture
0 likes · 13 min read
Data Architecture Trends: From Chaos to an Organized Era – Insights from Anthony J. Algmin