Understanding Data Mesh: Concepts, Evolution, and Challenges

During a weekend discussion about Data Mesh, the author revisited the term after first hearing it in early 2020, reviewing related papers and introductions to better understand the concept.

Before diving into Data Mesh, the article defines Service Mesh as a dedicated infrastructure layer that handles communication between services, essentially serving as a service governance platform that decouples business logic.

The evolution of service architectures is traced from monolithic modular designs, through Service‑Oriented Architecture (SOA), classic microservices that use RPC, and finally to Service Mesh, each step emphasizing greater decoupling and reuse.

Data Mesh is presented as a distributed, domain‑driven, self‑serve data architecture pattern that borrows ideas from microservices and Service Mesh.

The author outlines three phases of big‑data platform evolution: (1) enterprise data warehouses for BI, (2) data‑lake‑centric ecosystems, and (3) cloud‑native platforms that combine real‑time streaming, batch processing, cloud storage, pipelines, and machine‑learning capabilities.

Limitations of traditional approaches are highlighted, especially the high development and operational costs that often result in low ROI.

According to Data Mesh’s founder, the approach consists of a set of architectural principles that merge Distributed Domain‑Driven Architecture, self‑serve platform design, and the mindset of treating data as a product.

ThoughtWorks identifies four core components of Data Mesh: domain‑specific data or ML products, a self‑service data infrastructure, product‑oriented governance and roles, and a CI‑based delivery infrastructure.

The article notes that Data Mesh targets business teams rather than being exclusive to data teams, should be delivered as a service, and currently lacks concrete implementation examples.

Finally, the author expresses anticipation for ThoughtWorks to share practical Data Mesh scenarios and solutions.