Metadata Optimization

DataFunSummit

Apr 27, 2024 · Big Data

Delta Lake 3.1: New Features, Metadata Optimization, and Universal Format Overview

This article introduces Delta Lake 3.1, detailing its release background, the addition of Deletion Vector to Update and Merge commands, metadata‑driven count/min/max optimizations, the Universal Format for cross‑engine compatibility, and a comparative evaluation with Iceberg and Hudi.

Big DataDeletion VectorDelta Lake

0 likes · 8 min read

Delta Lake 3.1: New Features, Metadata Optimization, and Universal Format Overview

Tencent Cloud Developer

Feb 28, 2022 · Big Data

GooseFS: Distributed Caching System for Storage-Compute Separation Architecture

GooseFS, Tencent Cloud’s distributed caching system for storage‑compute separation, links compute frameworks to underlying storage (COS, CHDFS, COSN) and boosts big‑data and AI workloads by 2‑10× through transparent acceleration, robust master‑worker architecture, Raft‑based HA, tiered caching, and metadata optimizations, delivering up to 50% cost savings and 29% faster compute jobs.

GooseFSMetadata OptimizationRaft consensus

0 likes · 18 min read

GooseFS: Distributed Caching System for Storage-Compute Separation Architecture