Choosing the Right Data Architecture: Warehouse, Mart, or Lake?

Building an enterprise‑level data platform starts with understanding corporate data, confirming management needs, and selecting a suitable data management architecture. Faced with diverse sources and structures, organizations must decide where to begin and which architecture fits best. This article introduces data warehouses, data marts, and data lakes.

Data Warehouse (Data Warehouse)

According to Bill Inmon’s definition, a data warehouse is a subject‑oriented, integrated, non‑volatile, time‑variant collection of data that supports decision making.

It serves as a unified data management approach, aggregating data from various applications, processing it for multidimensional analysis, and presenting key performance indicators (KPIs) to support management decisions and trend forecasting. Thus, a data warehouse is a core system in enterprise IT.

Enterprise Data Warehouse

Enterprises consolidate internal (OLTP transaction systems and OLAP analytical systems) and external data into databases such as Teradata, Oracle, or DB2, then perform offline batch processing to build subject models and provide reporting.

Real‑time Data Warehouse

Some industries require real‑time analytics (e.g., retail inventory adjustment, wind‑farm fault detection). Traditional offline batch processing cannot meet these needs, so a real‑time data warehouse processes data within defined time windows using engines like Flink or Slipstream, enabling event‑driven analysis, machine learning, and real‑time scheduling.

Data Mart (Data Mart)

A data mart is a targeted subset of a data warehouse, containing data important to a specific team or user group. It enables faster, more focused insights compared to the broader warehouse and originated in the 1990s to address the difficulty of integrating enterprise‑wide data.

Because its scope is limited, a data mart is easier and quicker to implement, with lower infrastructure and security requirements, making it agile for departmental use.

Types of Data Marts

Independent Data Mart : Operates without relying on a warehouse or lake, loading necessary data directly from source systems, processing it, and delivering business analysis.

Associated Data Mart : Part of a warehouse or lake, with data processing handled by the warehouse’s batch jobs.

Hybrid Data Mart : Sources include the warehouse, lake, and other databases, combining top‑down curated data with bottom‑up analyst‑driven needs.

Data Lake (Data Lake)

A data lake is a storage repository that allows users to store structured and unstructured data at any scale, supporting rapid processing and analysis. Data can be ingested in its raw form and used for dashboards, visualizations, big‑data processing, real‑time analytics, and machine learning.

Data lakes were created to handle the volume, velocity, and variety of big data that traditional warehouses struggle with. They store raw data (often petabytes) without complex modeling, providing a public data store for exploration and serving downstream marts, warehouses, or middle‑platforms.

Core Capabilities of a Data Lake

Data Integration : Ability to ingest structured data from relational databases and unstructured data from the internet, documents, sensors, etc., supporting real‑time, near‑real‑time, and batch integration with transformation capabilities.

Data Computing : Provide powerful compute resources to retrieve key information from massive datasets, perform correlation analysis, and conduct deep analysis of unstructured data.

Data Governance : Tools to improve data quality and value within the lake, addressing the lack of pre‑modeled data and inconsistent source standards.

Data Service : Enable self‑service access, discovery, analysis, and contribution, with a robust data catalog and multi‑tenant capabilities.

Important Non‑Functional Requirements

Interoperability : Seamless interaction with relational databases, NoSQL stores, real‑time systems, and object storage.

Cost Control : Low hardware and operational costs, efficient resource utilization, elastic scaling, and metered billing.

Multi‑tenant : Support multiple departments with isolated resources while handling diverse workloads such as CPU‑intensive machine learning and I/O‑heavy queries.

Business Continuity : High availability and disaster‑recovery capabilities to ensure rapid recovery under extreme failures.

Conclusion

This article introduced data warehouses, data marts, and data lakes. In upcoming pieces, we will discuss how to design a sustainable, evolving technical architecture based on an enterprise’s digital maturity, covering five stages of platform construction.