FastData Real‑Time Intelligent Lakehouse Platform: Data Fabric Technology Practice

Data Fabric is an emerging data‑management design that enables seamless integration and sharing across heterogeneous data sources, reducing ETL effort and breaking data silos. Gartner has highlighted it as a top data‑analysis trend.

Based on Data Fabric, Dipu Technology developed FastData, a one‑stop real‑time intelligent lakehouse platform composed of three layers: the DLink engine for storage and compute across cloud infrastructures, a development suite offering scheduling, editors, and workflow orchestration, and an analysis suite that manages business metrics using a unified model language.

The platform follows a Modern Data Stack (MDS) approach, providing plug‑in‑style components that can be assembled as needed, thus lowering cost and simplifying architecture. Its storage layer uses Apache Iceberg tables with Flink CDC connectors for real‑time change capture, while compute workloads are handled by Spark (batch), Flink (streaming), and Trino (interactive queries).

FastData’s core advantages are low cost (cloud‑agnostic deployment on object storage), ease of use (low‑code development tools), modularity (plug‑in architecture), and extensibility (support for both open‑source and proprietary ecosystems). Automated table maintenance, materialized view refresh, and incremental processing further enhance performance.

Practical cases include accelerating data collection in oil fields from daily to minute‑level latency, building distributed lakehouses for centralized analytics while keeping data at local sites, and enabling retailers and new‑energy vehicle manufacturers to unify structured, semi‑structured, and unstructured data for better marketing and service insights.

Future plans focus on improving high‑concurrency performance, unifying gateway services for a MySQL‑like experience, expanding support for additional cloud environments, and leveraging large‑language models to automate data‑asset monetization, natural‑language query translation, and SQL generation.