Tagged articles
Schema Evolution
18 articles · Page 1 of 1
Ziroom migrated 4,400+ MySQL tables to a real-time lakehouse using Apache StreamPark and Paimon, replacing a complex Canal/Kafka/Flink/Airflow pipeline with standardized CDC ingestion, automated job deployment via StreamPark API, and Paimon's snapshot management to reduce source database load and operational overhead.
Apache FlinkApache PaimonApache StreamPark
0 likes · 35 min read

The author migrates a high-frequency internal inventory query from REST to gRPC in Spring Boot 4.1, discovering that gRPC's contract-first approach, binary encoding, and explicit timeout handling improve interface governance for service-to-service RPC, while REST remains better for web-facing and low-frequency APIs.
API ContractProtobufSchema Evolution
0 likes · 17 min read

This article details migrating from a legacy Canal+Kafka+Flink+Hive stack to a modern Flink CDC + Iceberg real-time data lake, covering Spring Boot orchestration, chunk-based full/incremental sync, Interval Join for wide tables, automated schema evolution, hidden partitioning, Exactly-Once checkpoint tuning, and Kubernetes deployment practices.
Apache IcebergFlink CDCKubernetes
0 likes · 17 min read


XikeAug 1, 2026 · Big Data
This tutorial explains how Apache Iceberg adds snapshot metadata to files on HDFS or object storage, enabling atomic writes, time‑travel queries, and schema evolution, and walks through a Docker‑based setup with a REST catalog, Spark 3.5.3, and hands‑on examples including table creation, data insertion, version queries, and troubleshooting tips.
Apache IcebergHDFSSchema Evolution
0 likes · 16 min read

This article details a production-verified zero-downtime database migration strategy using Spring Boot and Flyway, covering why DDL causes outages, Flyway configuration best practices, the expand-migrate-contract pattern for smooth schema changes, script writing and rollback techniques, Kubernetes deployment coordination, monitoring with circuit breakers, and a post-mortem checklist.
DDLDatabase MigrationFlyway
0 likes · 18 min read

In the AI era, enterprises need a data foundation that supports both low‑latency streaming and long‑term analytics, and the combination of Kafka, Iceberg and object storage is emerging as a preferred solution; by moving ingestion capabilities closer to the message layer and eliminating external ETL jobs, a "zero‑ETL" approach reduces architectural complexity, improves consistency, and streamlines schema evolution and small‑file management.
CDCKafkaSchema Evolution
0 likes · 27 min read

This article details how a leading Southeast Asian tech group migrated its real‑time write workloads from Google BigQuery to MaxCompute using MaxCompute Streaming Insert, covering architecture, core features, migration challenges, optimization strategies, business impact, and future enhancements.
BigQuery MigrationMaxComputeSchema Evolution
0 likes · 9 min read

This article compares five common database extension strategies—from simple MySQL column additions to a hybrid MySQL‑HBase solution—detailing their implementation, advantages, drawbacks, and ideal scenarios, helping architects select the most scalable and maintainable design for evolving business data requirements.
HBaseMySQLSchema Evolution
0 likes · 8 min read

This article shares a personal, experience‑driven overview of Apache Paimon, highlighting its design simplicity, key capabilities such as schema evolution, stream‑batch unified processing, primary‑key support, and closed‑loop data handling, while discussing when its features are appropriate for production environments.
Apache PaimonSchema Evolutionbatch processing
0 likes · 5 min read

Apache Iceberg offers a modern, ACID‑compliant table format for data lakes with features like hidden partitions and schema evolution, while StarRocks provides high‑performance query acceleration, metadata caching, and distributed planning to address Iceberg’s latency challenges, enabling seamless lake‑warehouse integration and real‑time analytics.
Apache IcebergMetadata CachingSchema Evolution
0 likes · 19 min read

This article presents OPPO's smart data lakehouse solution, describing its massive EB‑scale architecture, the integration of batch and streaming engines, the Glacier service for table management, schema‑adaptive ingestion, performance optimizations, and future technical road‑maps for unified data processing.
FlinkOPPOSchema Evolution
0 likes · 15 min read

This presentation introduces Iceberg's core capabilities, details Xiaomi's practical applications—including log ingestion, near‑real‑time warehousing, offline challenges, column‑level encryption, and Hive migration—and outlines future development directions such as materialized views and cloud migration, providing a comprehensive view of modern data‑lake engineering.
FlinkSchema EvolutionSpark
0 likes · 22 min read

This presentation details Iceberg's core capabilities—transactional writes, schema evolution, implicit partitioning, and row‑level updates—while showcasing Xiaomi's real‑world applications such as log ingestion redesign, near‑real‑time warehousing, offline optimizations, column‑level encryption, Hive migration strategies, and outlining upcoming enhancements like materialized views and cloud migration.
Column EncryptionFlinkSchema Evolution
0 likes · 20 min read

This article describes how ByteDance tackled petabyte‑scale feature storage by adopting Apache Iceberg, detailing the problem background, design choices, implementation of COW and MOR back‑fill strategies, performance optimizations, and future plans such as lake‑cold‑layering and materialized views.
Apache IcebergFeature StoreMOR
0 likes · 16 min read

This article presents an in‑depth overview of Apache Iceberg as used at Tencent, covering its table format architecture, Spark read/write mechanisms, production challenges and optimizations such as schema evolution, file filtering, upsert strategies, and the surrounding data‑governance services.
Apache IcebergSchema EvolutionSpark
0 likes · 19 min read


JavaEdgeJun 26, 2022 · Backend Development
This article explains why forward and backward compatibility are crucial for evolving systems, covering database encoding, schema evolution, REST and RPC communication, message brokers, and actor frameworks, and provides practical guidance for designing compatible data flows across services.
Message BrokerRESTRPC
0 likes · 22 min read

Fusion-NewSQL is Didi’s internally‑developed NewSQL system built atop the Fusion distributed KV store, offering MySQL compatibility, high throughput, low latency, flexible schema changes, secondary indexes, and integration with ElasticSearch and Hive, with detailed architecture, data flow, and future roadmap.
MySQL CompatibilityNewSQLSchema Evolution
0 likes · 16 min read

Fusion‑NewSQL is a NewSQL layer built atop Didi’s distributed KV store Fusion, translating MySQL queries into Redis‑style hashmaps, asynchronously maintaining secondary indexes, supporting fast Hive‑to‑Fusion loads and Elasticsearch integration, thereby delivering over 2 million QPS, 600 TB storage and flexible schema evolution for dozens of services.
MySQL CompatibilityNewSQLSchema Evolution
0 likes · 15 min read
