Focused on big data technology research, exploring the Big Data era | [email protected]
This article outlines the essential requirements, architectural design, and five key capabilities of an enterprise‑level unified data platform, explaining how data integration, cloud‑native features, and layered design enable digital transformation and business innovation.
Traditional companies face the urgent need to digitize, turning decades‑old assets like customer data and supply chains into online services by adopting data‑driven warehouses, cloud‑native microservices, and scalable cloud computing, enabling faster innovation, AI integration, and competitive advantage in the digital era.
This article explains the principles of Logical Undo Logging, compares it with Physical Logging, and examines advanced recovery techniques such as SQL Server's Constant Time Recovery and the in‑memory database Silo's Force Recovery, highlighting their impact on concurrency, performance, and recovery latency.
This article explains the essential role of logging and recovery mechanisms—especially the ARIES algorithm—in ensuring ACID properties, covering physical vs logical logging, optimization techniques like group commit and early lock release, and the three-phase recovery process.
This article reviews the challenges of scheduling heterogeneous workloads—micro‑services, big‑data, AI, and HPC—on a unified cloud‑native platform, compares existing schedulers like Mesos and YARN, examines Kubernetes ecosystem extensions such as Volcano and YuniKorn, and details the design and components of the Transwarp Scheduler built on Kubernetes Scheduling Framework v2.
This article explores the concurrency control strategies, persistence mechanisms, and compiled query processing techniques used by modern memory databases, comparing systems like Hekaton, Hyper, HANA, and others while highlighting performance trade‑offs and architectural considerations.
This article explains how modern in‑memory database systems structure data into fixed‑ and variable‑length blocks, choose partitioned or non‑partitioned architectures, handle row versus column storage, and implement cache‑aware index structures such as CSB+-Tree, PB+-Tree, Bw‑Tree, Adaptive Radix, OLFIT and Skiplists, illustrated with Hekaton, H‑Store/VoltDB, HyPer and SAP HANA examples.
This article examines the fundamental differences between traditional disk‑based DBMS and modern in‑memory databases, covering buffer management, lock versus latch mechanisms, logging and recovery, performance overhead, historical evolution, and architectural innovations that enable high‑performance memory‑resident data processing.
This article explores the concept of data quality, identifies ten common root causes, presents a comprehensive data quality management framework, outlines evaluation methods and key dimensions, and discusses future challenges and tools for improving data quality in large‑scale data environments.
This article explains why Kubernetes requires an overlay network, describes Docker’s virtual bridge limitations, introduces Flannel’s architecture and its three data‑forwarding modes—UDP, VXLAN, and host‑gw—detailing their mechanisms, performance trade‑offs, and how the StarRing TCOS platform leverages these modes for different cluster sizes and network topologies.
