BestHub
Sign in
Tag

DataMigration

1 views collected around this technical thread.

Aikesheng Open Source Community
Aikesheng Open Source Community
Dec 10, 2024 · Databases

Random Data Source Selection Issue When Adding Replicas in OceanBase 4.2.1.4 and Its Solutions

An OceanBase 4.2.1.4 deployment experienced random selection of data sources when adding tenant replicas, causing cross‑city bandwidth congestion; the article reproduces the issue, analyzes its cause, and presents three remediation methods—including setting the choose_migration_source_policy parameter, manual replica addition, and temporary network isolation.

DataMigrationDatabaseConfigurationDisasterRecovery
0 likes · 9 min read
Random Data Source Selection Issue When Adding Replicas in OceanBase 4.2.1.4 and Its Solutions
Top Architect
Top Architect
Nov 7, 2024 · Databases

Database Sharding Design and Implementation for Loan & Repayment Tables Using ShardingSphere and Spring Boot

This article describes how a senior architect designed and implemented a sharding solution for loan and repayment tables, covering database design, table naming, sharding algorithms, historical data migration, three‑write synchronization, dynamic switches, and the necessary Spring Boot, MyBatis‑Plus, and ShardingSphere configurations with code examples.

DataMigrationDatabaseMySQL
0 likes · 22 min read
Database Sharding Design and Implementation for Loan & Repayment Tables Using ShardingSphere and Spring Boot
Didi Tech
Didi Tech
Jan 25, 2021 · Big Data

Migrating Hive SQL to Spark SQL: Design, Implementation, and Performance Evaluation at DiDi

DiDi migrated over 10,000 Hive SQL tasks to Spark SQL using a lightweight dual‑run pipeline that extracts, rewrites, compares, and switches tasks, fixing syntax and UDF differences while adding features such as small‑file merging and enhanced partition pruning, resulting in Spark handling 85 % of workloads with 40 % faster execution, 21 % lower CPU and 49 % lower memory usage.

BigDataDataMigrationHive
0 likes · 18 min read
Migrating Hive SQL to Spark SQL: Design, Implementation, and Performance Evaluation at DiDi
Qunar Tech Salon
Qunar Tech Salon
Jul 8, 2020 · Databases

Handling Foreign Key Constraint Issues During Data Migration in PostgreSQL/Greenplum

This article explains how to resolve foreign key constraint violations that occur when migrating data from multiple source databases into a target PostgreSQL/Greenplum database by disabling, dropping, or deferring constraints and then re‑enabling them to ensure full data integrity.

ConstraintManagementDataMigrationDatabase
0 likes · 4 min read
Handling Foreign Key Constraint Issues During Data Migration in PostgreSQL/Greenplum