Tagged articles
252 articles
Page 1 of 3
Java Architect Essentials
Java Architect Essentials
May 11, 2026 · Databases

Why SQLark Beats Navicat: An AI‑Powered Database Tool Review

The author compares common database clients, highlights the risks of cracked Navicat and limited DBeaver, then details how SQLark’s cross‑platform support, AI‑driven features, smart SQL editor, data migration, ER diagram generation, and test‑data generator provide a more efficient and secure workflow for developers.

AI AssistantData MigrationDatabase Management
0 likes · 7 min read
Why SQLark Beats Navicat: An AI‑Powered Database Tool Review
Java Baker
Java Baker
Apr 22, 2026 · Databases

A Step‑by‑Step SOP for Seamless Business Data Migration

This article outlines a comprehensive, risk‑controlled SOP for migrating business data—including model changes, storage shifts, incremental dual‑write, back‑filling, full and incremental consistency checks, read‑switching, and final decommissioning—backed by concrete SQL examples and visual diagrams.

BackendData ConsistencyData Migration
0 likes · 6 min read
A Step‑by‑Step SOP for Seamless Business Data Migration
Alibaba Cloud Observability
Alibaba Cloud Observability
Mar 9, 2026 · Cloud Native

How LoongCollector’s One‑Time File Collection Simplifies Bulk Log Migration

LoongCollector introduces a One‑Time file collection mode that scans matching files once, records a snapshot, and exits, enabling efficient historic log migration, data back‑fill, and temporary batch processing while providing fine‑grained checkpoints, execution windows, and throttling controls to avoid quota issues and ensure reliable completion.

CheckpointData Migrationlog collection
0 likes · 12 min read
How LoongCollector’s One‑Time File Collection Simplifies Bulk Log Migration
Java Companion
Java Companion
Feb 1, 2026 · Databases

How I Completed a 100+ Table Migration in One Month Using Navicat Tricks

In under a month the author migrated more than 100 heterogeneous tables from MySQL and MongoDB to PostgreSQL across isolated networks by automating Navicat's import/export logic, streaming data, handling special characters, configuring flexible sync strategies, and using OSS as a bridge to avoid OOM and lock‑contention.

Data MigrationMongoDBNavicat
0 likes · 20 min read
How I Completed a 100+ Table Migration in One Month Using Navicat Tricks
ITPUB
ITPUB
Jan 15, 2026 · Databases

How to Migrate ClickHouse Data to Doris: Three Practical Strategies Tested

Facing a ClickHouse cluster shutdown, the author explores three migration methods—using Doris’s ClickHouse catalog, exporting to files with Broker/Stream Load, and Spark—to transfer ~10 billion rows to Doris, evaluating each for simplicity, bugs, and performance, and sharing detailed steps, code snippets, and benchmark results.

ClickHouseData MigrationSQL
0 likes · 9 min read
How to Migrate ClickHouse Data to Doris: Three Practical Strategies Tested
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Jan 12, 2026 · Databases

Designing Scalable Order Sharding for Millions of Daily Transactions

This article outlines a practical sharding strategy for e‑commerce order systems, estimating future load, detailing user‑centric partitioning, heterogeneous designs for merchants and operators, and migration steps to achieve high‑concurrency writes and massive storage without downtime.

Data MigrationDistributed SystemsOrder Management
0 likes · 4 min read
Designing Scalable Order Sharding for Millions of Daily Transactions
IT Services Circle
IT Services Circle
Dec 29, 2025 · Backend Development

Mastering Sharding: When, How, and How Much to Split Your Database

This guide walks senior backend engineers through the strategic reasoning, capacity estimation, and step‑by‑step migration techniques for sharding databases, covering when to split, choosing between partitioning, read/write splitting, or full sharding, and how to plan safe expansions.

Data MigrationPartitioningcapacity planning
0 likes · 19 min read
Mastering Sharding: When, How, and How Much to Split Your Database
JD Retail Technology
JD Retail Technology
Dec 25, 2025 · Backend Development

How We Rebuilt a 15‑Year‑Old Review Platform: From Monolithic Code to a Scalable DDD‑Driven Architecture

This article details the complete redesign of a fifteen‑year‑old e‑commerce review system, covering its legacy pain points, the strategic choice of a full‑stack reconstruction using Domain‑Driven Design, the new layered micro‑service architecture, data migration tactics, operational challenges, organizational safeguards, and the measurable performance gains achieved after launch.

BackendDDDData Migration
0 likes · 35 min read
How We Rebuilt a 15‑Year‑Old Review Platform: From Monolithic Code to a Scalable DDD‑Driven Architecture
Ctrip Technology
Ctrip Technology
Nov 27, 2025 · Big Data

How Ctrip Cut Query Latency by 85% with StarRocks’ Compute‑Storage Separation

Ctrip migrated its massive User Behavior Tracking system from ClickHouse to a compute‑storage separated StarRocks cluster on Kubernetes, achieving millisecond‑level query latency, halving storage usage, reducing node count, and sustaining millions‑of‑rows‑per‑second write throughput while simplifying scaling and operations.

Big DataClickHouseCompute-Storage Separation
0 likes · 15 min read
How Ctrip Cut Query Latency by 85% with StarRocks’ Compute‑Storage Separation
Old Meng AI Explorer
Old Meng AI Explorer
Nov 24, 2025 · Databases

Why DBeaver Is the Ultimate All‑In‑One Database Management Tool

Developers, data analysts, and ops engineers can eliminate the hassle of juggling multiple database clients by adopting DBeaver, an open‑source, cross‑platform tool that supports over 100 databases, offers powerful SQL editing, visual design, data migration, and backup features—all for free.

DBeaverData MigrationDatabase Management
0 likes · 11 min read
Why DBeaver Is the Ultimate All‑In‑One Database Management Tool
Alibaba Cloud Developer
Alibaba Cloud Developer
Nov 20, 2025 · Big Data

Mastering Large‑Scale Data Migration: Challenges, Strategies and Real‑World Solutions

This article explains why data migration is the essential first step for cloud modernization, outlines the technical challenges of moving terabytes to petabytes, compares physical and logical migration methods, and presents practical solutions and real‑world case studies across Hive, cloud warehouses, lake‑house formats and analytic databases.

Big DataData MigrationETL
0 likes · 56 min read
Mastering Large‑Scale Data Migration: Challenges, Strategies and Real‑World Solutions
Java Captain
Java Captain
Oct 29, 2025 · Databases

How to Seamlessly Migrate MongoDB to an Isolated Instance with Real-Time Sync

This guide details a step‑by‑step process for migrating a high‑traffic MongoDB app to a dedicated instance, covering full backup, real‑time incremental sync via Change Streams, compensating for missed changes, handling operation ordering, and ensuring zero downtime throughout.

BackupChange StreamsData Migration
0 likes · 11 min read
How to Seamlessly Migrate MongoDB to an Isolated Instance with Real-Time Sync
StarRocks
StarRocks
Oct 14, 2025 · Big Data

How Ctrip Scaled UBT Analytics by Migrating from ClickHouse to StarRocks

Ctrip's User Behavior Tracking (UBT) system, handling 30 TB of daily data, moved from ClickHouse to StarRocks' compute‑storage separated architecture, cutting average query latency from 1.4 seconds to 203 ms, halving storage, reducing nodes from 50 to 40, and boosting write throughput to 3 million rows per second.

Big DataClickHouseData Migration
0 likes · 15 min read
How Ctrip Scaled UBT Analytics by Migrating from ClickHouse to StarRocks
Mingyi World Elasticsearch
Mingyi World Elasticsearch
Oct 12, 2025 · Backend Development

5 Practical Ways to Migrate Elasticsearch Data to Easysearch

This article evaluates five common methods for moving data from cloud Elasticsearch to on‑premise INFINI Easysearch, details three viable solutions—Logstash, Elasticdump, and custom Python scripts—explains why reindex and snapshot restore fail, and provides a decision matrix and practical migration guidelines.

Data MigrationEasysearchElasticdump
0 likes · 7 min read
5 Practical Ways to Migrate Elasticsearch Data to Easysearch
Mingyi World Elasticsearch
Mingyi World Elasticsearch
Oct 8, 2025 · Operations

Step‑by‑Step Logstash Migration from Elasticsearch 9.0 to Easysearch with Zero Downtime

This guide walks through a zero‑downtime migration from Elasticsearch 9.0 to the Easysearch compatible branch using Logstash, covering project goals, solution selection, detailed input/filter/output configurations, SSL handling, batch tuning, execution steps, and verification to ensure data integrity and ID consistency.

Data MigrationEasysearchElasticsearch
0 likes · 6 min read
Step‑by‑Step Logstash Migration from Elasticsearch 9.0 to Easysearch with Zero Downtime
Huolala Tech
Huolala Tech
Sep 26, 2025 · Big Data

How We Migrated 40 PB of Hive Data Across Clouds with Zero Downtime

This article details the end‑to‑end design, challenges, and implementation of a cross‑cloud migration of over 200 k Hive tables and nearly 40 PB of data using the self‑developed Kirk service, covering architecture, verification steps, and lessons learned to achieve 100 % data consistency without impacting production services.

Big DataData ConsistencyData Migration
0 likes · 20 min read
How We Migrated 40 PB of Hive Data Across Clouds with Zero Downtime
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Sep 19, 2025 · Big Data

How MMS Powered a 50 PB BigQuery‑to‑MaxCompute Migration for GoTerra

This article details GoTerra's massive six‑month, 50 PB migration from GCP BigQuery to Alibaba Cloud MaxCompute, covering project scope, technical challenges such as complex data types, partition strategies, and high‑speed requirements, and explaining how the MaxCompute Migration Service (MMS) solved them with innovative architecture, scheduling, and data‑reorder techniques.

Big DataBigQueryData Migration
0 likes · 14 min read
How MMS Powered a 50 PB BigQuery‑to‑MaxCompute Migration for GoTerra
Tech Freedom Circle
Tech Freedom Circle
Sep 11, 2025 · Backend Development

How to Optimize a Redis Big Key Online Without Disrupting Existing Services (Interview Answer)

The article explains what a Redis big key is, why it harms performance, typical scenarios that generate big keys, and provides a step‑by‑step online optimization plan—including key sharding, dual‑write synchronization, progressive migration with HSCAN, gray‑scale traffic switch, non‑blocking deletion using UNLINK, and monitoring with rollback procedures—to answer the interview question confidently.

BackendBig KeyData Migration
0 likes · 16 min read
How to Optimize a Redis Big Key Online Without Disrupting Existing Services (Interview Answer)
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Sep 10, 2025 · Big Data

Unlock Seamless BigQuery to MaxCompute Migration with dbt‑maxcompute

This article details the real‑world migration of Southeast Asian tech leader GoTerra from BigQuery to MaxCompute, showcasing how the open‑source dbt‑maxcompute adapter enables smooth ELT transitions, advanced incremental strategies, performance gains, ecosystem compatibility, and comprehensive best‑practice implementations for large‑scale data pipelines.

Big DataData MigrationELT
0 likes · 13 min read
Unlock Seamless BigQuery to MaxCompute Migration with dbt‑maxcompute
php Courses
php Courses
Sep 1, 2025 · Backend Development

How to Migrate Complex PHP Data Safely While Minimizing Downtime

This guide outlines essential strategies for PHP developers to execute complex data migrations—covering thorough planning, incremental migration, transaction safety, dual‑write architectures, feature‑flag rollouts, real‑time syncing, and comprehensive monitoring—to ensure data integrity and keep system downtime to a minimum.

Backend DevelopmentData MigrationPHP
0 likes · 7 min read
How to Migrate Complex PHP Data Safely While Minimizing Downtime
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Aug 15, 2025 · Big Data

How MaxCompute Extends SQL to Match BigQuery – New Features & Compatibility

This article details the challenges of migrating 100,000+ SQL statements from BigQuery to MaxCompute for a leading Southeast Asian tech group, explains the new MaxCompute SQL syntax, auto‑partition and ingestion‑time tables, enhanced built‑in functions, and the BigQuery compatibility mode that ensures seamless query behavior.

BigQueryCompatibilityData Migration
0 likes · 15 min read
How MaxCompute Extends SQL to Match BigQuery – New Features & Compatibility
Top Architect
Top Architect
Jul 24, 2025 · Databases

How to Efficiently Shard Billion‑Row Tables with ShardingSphere and Spring Boot

This article walks through the end‑to‑end design, configuration, and implementation of splitting massive loan and repayment tables into 50 sharded MySQL tables using ShardingSphere, Spring Boot, and a custom suffix algorithm, while covering data migration, DBA coordination, dynamic switches, and scheduled consistency checks.

Data MigrationShardingSphereSpring Boot
0 likes · 19 min read
How to Efficiently Shard Billion‑Row Tables with ShardingSphere and Spring Boot
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Jul 15, 2025 · Big Data

How MaxCompute’s Append DeltaTable Transforms BigQuery Migration

This article details the complex migration of a leading Southeast Asian tech group's data warehouse from Google BigQuery to Alibaba Cloud MaxCompute, outlining challenges such as storage format differences, SQL compatibility, and performance tuning, and explains how the new Append DeltaTable format with dynamic bucketing and incremental reclustering resolves these issues.

Big DataData MigrationData Warehouse
0 likes · 19 min read
How MaxCompute’s Append DeltaTable Transforms BigQuery Migration
Tech Freedom Circle
Tech Freedom Circle
Jul 14, 2025 · Databases

How to Estimate Sharding Capacity: Calculating Required Databases and Tables for an Alibaba Interview

The article walks through why sharding is needed, outlines IO and CPU bottlenecks, presents two design principles, shows how to estimate capacity from existing data and growth trends, compares range, modulo, consistent‑hash and Snowflake sharding schemes, and details migration strategies for expanding nodes without downtime.

Data Migrationcapacity planningconsistent hashing
0 likes · 22 min read
How to Estimate Sharding Capacity: Calculating Required Databases and Tables for an Alibaba Interview
Selected Java Interview Questions
Selected Java Interview Questions
Jul 8, 2025 · Databases

How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration

This article explores the performance bottlenecks of a 700‑million‑row MySQL orders table and presents a comprehensive sharding solution—including vertical and horizontal partitioning, gene‑based shard keys, routing logic, global secondary indexes, data migration strategies, and common pitfalls—demonstrating how query latency can drop from seconds to milliseconds while supporting billions of records.

Data MigrationPartitioningdatabase scaling
0 likes · 9 min read
How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
Jul 1, 2025 · Backend Development

How We Cut a 150‑Billion Image Migration from 120 Days to 40 Days

Facing the challenge of moving 15 billion image files from Cassandra to S3, we iteratively redesigned the migration pipeline—from a single‑process approach to multi‑process, queue‑driven, and multi‑cluster deployments—reducing the projected 120‑day effort to just 40 days while ensuring reliability and performance.

Data MigrationGoPerformance Optimization
0 likes · 16 min read
How We Cut a 150‑Billion Image Migration from 120 Days to 40 Days
Top Architect
Top Architect
Jun 27, 2025 · Databases

How to Implement High‑Performance Database Sharding with ShardingSphere and SpringBoot

This article walks through the complete process of designing, configuring, and coding a sharding solution for loan and repayment tables—including database basics, historical data migration, backend query refactoring, a three‑write strategy, scheduled consistency checks, and full SpringBoot/ShardingSphere implementation details.

Data MigrationMyBatisShardingSphere
0 likes · 20 min read
How to Implement High‑Performance Database Sharding with ShardingSphere and SpringBoot
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Apr 30, 2025 · Databases

How to Reshard Slots in a Redis Cluster

This article explains the process of moving hash slots between nodes in a Redis cluster using the redis-cli --cluster reshard command, detailing the required parameters, interactive steps, and verification of successful slot migration.

ClusterData MigrationResharding
0 likes · 6 min read
How to Reshard Slots in a Redis Cluster
Architect
Architect
Apr 29, 2025 · Databases

Lessons Learned from Misusing INSERT INTO SELECT in MySQL Data Migration

A real‑world MySQL data‑migration case study shows how using INSERT INTO SELECT without proper indexing caused a full table scan, OOM errors, and data loss, and explains how to avoid the pitfall by adding appropriate indexes and understanding transaction locking.

Data MigrationDatabase PerformanceFull Table Scan
0 likes · 8 min read
Lessons Learned from Misusing INSERT INTO SELECT in MySQL Data Migration
Su San Talks Tech
Su San Talks Tech
Apr 29, 2025 · Databases

Why Split Databases? Master Sharding Concepts, Strategies, and Practical SQL Routing

This article explains the fundamental concepts of database sharding—including data nodes, logical and broadcast tables, sharding keys, strategies, algorithms, SQL parsing, routing, rewriting, execution, result merging, distributed primary keys, data masking, distributed transactions, data migration, and shadow databases—providing clear examples and code snippets for real‑world implementation.

Data MigrationDistributed TransactionsSQL Routing
0 likes · 21 min read
Why Split Databases? Master Sharding Concepts, Strategies, and Practical SQL Routing
dbaplus Community
dbaplus Community
Apr 10, 2025 · Databases

How to Seamlessly Migrate MongoDB to New Data Stores Without Downtime

This article presents a complete, step‑by‑step plan for migrating a legacy MongoDB‑based system to alternative data stores—including MySQL, Elasticsearch, and JD’s JImKV—while ensuring zero service interruption through careful scope analysis, DAO refactoring, dual‑write synchronization, gray‑release rollout, and robust monitoring and rollback mechanisms.

Data MigrationDatabase RefactoringIncremental Sync
0 likes · 7 min read
How to Seamlessly Migrate MongoDB to New Data Stores Without Downtime
JD Tech
JD Tech
Jan 23, 2025 · Databases

Comprehensive Migration Plan for MongoDB to Alternative Data Stores

This article presents a complete MongoDB migration solution, detailing the migration rhythm, code refactoring using a decorator pattern, data source replacement with JImKV, MySQL and ES, bulk and incremental data transfer strategies, and deployment safeguards such as monitoring, gray release, and rollback to ensure a seamless cut‑over without service disruption.

Data MigrationDecorator PatternDistributed Systems
0 likes · 8 min read
Comprehensive Migration Plan for MongoDB to Alternative Data Stores
ITPUB
ITPUB
Jan 2, 2025 · Databases

How OceanBase Partitioning Supercharged KPOS Report Performance

This article details the background, challenges, and technical solutions behind the partitioning redesign of the KPOS reporting system on OceanBase, covering deployment choices, parameter tuning, data migration, phased rollout, performance gains, encountered issues, and future scaling plans.

Data MigrationDeployment StrategiesOceanBase
0 likes · 26 min read
How OceanBase Partitioning Supercharged KPOS Report Performance
Lobster Programming
Lobster Programming
Dec 26, 2024 · Databases

How to Migrate Tens of Millions of Rows: Strategies and Practical Steps

This article explains common data‑migration strategies such as hash‑modulo, time‑range and ID‑range sharding, and compares practical migration solutions including downtime migration, dual‑write, and an MQ + Redis approach, with detailed workflow diagrams and implementation tips.

Data MigrationMessage Queuedatabase scaling
0 likes · 8 min read
How to Migrate Tens of Millions of Rows: Strategies and Practical Steps
Xiaolei Talks DB
Xiaolei Talks DB
Dec 6, 2024 · Databases

Unlocking OceanBase: How Its Open‑Source Ecosystem Powers Modern Databases

OceanBase’s open‑source ecosystem—spanning data migration tools, developer utilities, a database proxy, and robust operational management—delivers comprehensive, open, and compatible support that enhances development, migration, and maintenance, positioning it as a competitive enterprise‑grade distributed relational database.

Data MigrationDatabase EcosystemHTAP
0 likes · 8 min read
Unlocking OceanBase: How Its Open‑Source Ecosystem Powers Modern Databases
Zhihu Tech Column
Zhihu Tech Column
Oct 28, 2024 · Databases

TiDB Architecture, Deployment Choices, and Zhihu’s Database Platform Practices

This article explains TiDB’s distributed relational database architecture, compares Kubernetes‑based and physical‑machine deployments, and describes Zhihu’s platform‑driven approach—including TiDB Operator, TiUP, self‑service tooling, monitoring, backup, security, and FinOps—to enable efficient database management for both developers and DBAs.

Data MigrationDatabase operationsFinOps
0 likes · 15 min read
TiDB Architecture, Deployment Choices, and Zhihu’s Database Platform Practices
Test Development Learning Exchange
Test Development Learning Exchange
Oct 24, 2024 · Databases

Comprehensive Testing of Business Data Migration in Enterprise Applications

This article details the comprehensive process of testing business data migration in enterprise applications, covering requirements analysis, environment preparation, test planning, data validation, performance testing, security checks, rollback strategies, and feedback mechanisms to ensure accuracy and integrity.

Backend DevelopmentData Migrationdatabase
0 likes · 8 min read
Comprehensive Testing of Business Data Migration in Enterprise Applications
DataFunSummit
DataFunSummit
Oct 17, 2024 · Big Data

Waggle Dance Based Metadata Solution at Tongcheng Travel: Architecture, Migration Strategies, and Future Outlook

This article presents Tongcheng Travel's metadata solution built on the open‑source Waggle Dance project, detailing the three‑layer architecture, challenges of a monolithic Hive Metastore, evaluated migration plans, federation implementation, migration workflow, and future directions for unified metadata governance.

Data MigrationFederationHive Metastore
0 likes · 11 min read
Waggle Dance Based Metadata Solution at Tongcheng Travel: Architecture, Migration Strategies, and Future Outlook
Selected Java Interview Questions
Selected Java Interview Questions
Oct 16, 2024 · Databases

Design and Implementation of Table Sharding for Loan and Repayment Applications Using ShardingSphere and SpringBoot

This article describes the end‑to‑end design, configuration, and code implementation of a 50‑table sharding solution for loan and repayment request data, covering database schema planning, historical data migration, backend query adaptation, DBA coordination, SpringBoot 3 integration, common pitfalls, and a complete demo.

Data Migration
0 likes · 18 min read
Design and Implementation of Table Sharding for Loan and Repayment Applications Using ShardingSphere and SpringBoot
Top Architect
Top Architect
Sep 17, 2024 · Databases

Design and Implementation of Database Table Sharding for Loan and Repayment Applications Using ShardingSphere and Spring Boot

This technical article describes how a senior architect designed, configured, and migrated loan and repayment application tables to a sharded MySQL setup using ShardingSphere, Spring Boot, and MyBatis‑Plus, covering schema design, historical data sync, dual‑write switches, code integration, and common pitfalls.

BackendData Migrationsharding
0 likes · 19 min read
Design and Implementation of Database Table Sharding for Loan and Repayment Applications Using ShardingSphere and Spring Boot
dbaplus Community
dbaplus Community
Sep 5, 2024 · Databases

How to Migrate Data from MongoDB to MySQL Using DuckDB

This guide explains how to export MongoDB collections to JSON, load them into DuckDB, generate compatible table schemas, and then transfer the data efficiently into MySQL using DuckDB as an intermediate processing engine.

Data MigrationDuckDBETL
0 likes · 6 min read
How to Migrate Data from MongoDB to MySQL Using DuckDB
Sohu Tech Products
Sohu Tech Products
Aug 28, 2024 · Databases

MySQL Data Migration with Double‑Write Using MyBatis Plugin

The article details how the vivo Game Center’s appointment service migrated billions of rows to a dedicated MySQL database with zero‑downtime by implementing a double‑write strategy via a custom MyBatis interceptor that mirrors updates and queries to the new database, handling full and incremental sync, primary‑key consistency, transaction limits, and final cut‑over steps.

Data MigrationMyBatisdatabase
0 likes · 19 min read
MySQL Data Migration with Double‑Write Using MyBatis Plugin
Aikesheng Open Source Community
Aikesheng Open Source Community
Aug 22, 2024 · Databases

ActionOMS: Intelligent Performance Diagnosis for OceanBase Data Migration

The article introduces ActionOMS, a customized version of OceanBase Migration Service that adds automated performance and fault diagnosis, explains its architecture, showcases a real‑world case of Oracle‑to‑OceanBase synchronization, and demonstrates how the tool improves migration throughput and reduces latency.

ActionOMSData MigrationDatabase Synchronization
0 likes · 11 min read
ActionOMS: Intelligent Performance Diagnosis for OceanBase Data Migration
vivo Internet Technology
vivo Internet Technology
Aug 21, 2024 · Databases

MySQL Data Migration with Double‑Write Strategy Using a MyBatis Plugin

The article details a zero‑downtime migration of a high‑traffic reservation service from a shared MySQL instance to an isolated database by employing a double‑write strategy implemented through a custom MyBatis plugin, covering full and incremental sync, consistency verification, traffic shifting, and key lessons on primary‑key handling and asynchronous write side‑effects.

Data MigrationMyBatisSynchronization
0 likes · 19 min read
MySQL Data Migration with Double‑Write Strategy Using a MyBatis Plugin
Top Architect
Top Architect
Aug 18, 2024 · Databases

Design and Implementation of Database Table Partitioning for Loan and Repayment Systems

This article details the design, planning, and step‑by‑step implementation of a database table‑partitioning solution for loan and repayment request tables, covering schema decisions, sharding strategy, historical data migration, dynamic synchronization, code examples with ShardingSphere, SpringBoot, and operational considerations for seamless migration.

Data MigrationShardingSphereSpringBoot
0 likes · 18 min read
Design and Implementation of Database Table Partitioning for Loan and Repayment Systems
Top Architect
Top Architect
Aug 9, 2024 · Databases

Design and Implementation of Database Sharding for Loan and Repayment Tables Using ShardingSphere and Spring Boot

After a year of business coding, the author details the design, implementation, and migration strategy for splitting loan and repayment tables into 50 sharded tables using ShardingSphere with Spring Boot, covering schema design, data synchronization, dynamic switches, and practical pitfalls.

Backend DevelopmentData MigrationSpring Boot
0 likes · 20 min read
Design and Implementation of Database Sharding for Loan and Repayment Tables Using ShardingSphere and Spring Boot
Wukong Talks Architecture
Wukong Talks Architecture
Aug 6, 2024 · Databases

Migrating Tencent Music's Data Infrastructure from ClickHouse and Druid to StarRocks: Strategy, Implementation, and Best Practices

This article details how Tencent Music’s data‑infrastructure team migrated thousands of ClickHouse and Druid nodes to a StarRocks compute‑storage‑separated lakehouse, achieving 40‑50% cost reduction while maintaining query performance, and shares the technical challenges, solutions, and best‑practice recommendations gathered during the process.

ClickHouseCost reductionData Migration
0 likes · 19 min read
Migrating Tencent Music's Data Infrastructure from ClickHouse and Druid to StarRocks: Strategy, Implementation, and Best Practices
JD Tech Talk
JD Tech Talk
Jul 30, 2024 · Databases

Database Performance Optimization and Governance for a High‑Load Application

This technical report analyzes a high‑traffic MySQL deployment with three servers, detailing disk usage, table space, QPS and slow‑SQL issues, and presents a comprehensive governance plan that includes data migration, read‑traffic offloading to replicas, MyBatis interceptor implementation, JSF monitoring, and Python automation to reduce disk pressure and improve query performance.

Data MigrationDatabase Optimizationmysql
0 likes · 18 min read
Database Performance Optimization and Governance for a High‑Load Application
JD Tech
JD Tech
Jul 26, 2024 · Databases

Database Performance Governance and Optimization for a High‑Load MySQL Application

This technical report describes the current high‑resource MySQL deployment, analyzes disk, table‑space, QPS and slow‑SQL issues, and presents a set of governance goals and concrete solutions—including data migration, query interception, and read‑from‑slave strategies—backed by code samples, tables and scripts to reduce load and improve stability before a major sales event.

Data MigrationQPS Reductionperformance tuning
0 likes · 23 min read
Database Performance Governance and Optimization for a High‑Load MySQL Application
Architect
Architect
Jul 17, 2024 · Databases

Design and Implementation of Table Sharding for Cash Repayment System Using ShardingSphere and SpringBoot

The article describes how a one‑year‑old loan/repayment service was refactored by introducing a 50‑table sharding scheme with ShardingSphere, detailing the design decisions, historical data migration, code changes, configuration files, pitfalls, and runtime synchronization strategies to achieve efficient query performance on billions of rows.

Backend DevelopmentData MigrationShardingSphere
0 likes · 23 min read
Design and Implementation of Table Sharding for Cash Repayment System Using ShardingSphere and SpringBoot
DeWu Technology
DeWu Technology
Jul 5, 2024 · Databases

StarRocks 2.5.13 Cross-Cluster Upgrade and Data Migration Practices

The article outlines a cross‑cluster upgrade to StarRocks 2.5.13, evaluating resource and stability costs, and presents two migration schemes—using external tables and a Flink connector—along with planning, parallel execution, validation steps, and results showing successful migration of over 10 TB at 2 Gb/s across ten nodes, while noting future automation and CDC enhancements.

Cluster UpgradeData MigrationExternal Table
0 likes · 15 min read
StarRocks 2.5.13 Cross-Cluster Upgrade and Data Migration Practices
dbaplus Community
dbaplus Community
Jun 16, 2024 · Backend Development

Building a Scalable MySQL Data Migration Service with Microservices and Design Patterns

This article presents a comprehensive guide to designing and implementing a Java‑based MySQL large‑table migration tool using microservice architecture, multiple design patterns, multithreading, custom logging, and flexible migration modes, complete with code snippets, repository links, and deployment considerations.

Data MigrationDesign PatternsJava
0 likes · 22 min read
Building a Scalable MySQL Data Migration Service with Microservices and Design Patterns
ZhongAn Tech Team
ZhongAn Tech Team
Jun 4, 2024 · Databases

Elasticsearch 8.x Performance Boosts, New Features, and Migration Guide

This article details how upgrading from Elasticsearch 5.x/2.x to 8.x dramatically improves search, aggregation, and write performance while reducing storage costs, introduces vector KNN, synthetic _source, TSDS, searchable snapshots, security enhancements, and provides migration examples and code snippets for enterprise search platforms.

Data MigrationElasticsearchHybrid Ranking
0 likes · 18 min read
Elasticsearch 8.x Performance Boosts, New Features, and Migration Guide
Top Architect
Top Architect
May 28, 2024 · Databases

Lessons Learned from Using INSERT INTO SELECT for MySQL Data Migration

This article recounts a real-world MySQL data migration incident where using INSERT INTO SELECT caused out‑of‑memory errors and data loss, analyzes why full table scans and transaction locking led to failures, and offers practical recommendations such as indexing and avoiding bulk inserts for large tables.

Data MigrationDatabase PerformanceFull Table Scan
0 likes · 8 min read
Lessons Learned from Using INSERT INTO SELECT for MySQL Data Migration
Architect
Architect
May 17, 2024 · Databases

Why ‘INSERT INTO SELECT’ Can Crash Your MySQL Migration and How to Fix It

A large‑scale MySQL table migration using INSERT INTO SELECT caused hidden full‑table scans and lock contention, leading to data loss and a costly OOM incident, but adding the proper index and understanding transaction isolation prevented the failure.

Data MigrationDatabase PerformanceINSERT INTO SELECT
0 likes · 8 min read
Why ‘INSERT INTO SELECT’ Can Crash Your MySQL Migration and How to Fix It
Java Captain
Java Captain
May 8, 2024 · Databases

Lessons Learned from Misusing INSERT INTO SELECT in MySQL Data Migration

A real‑world MySQL case study shows how using INSERT INTO SELECT for large‑scale data migration without proper indexing caused full‑table scans, lock contention, and data loss, highlighting the need for careful query planning and performance testing.

Data MigrationINSERT INTO SELECTdatabase
0 likes · 6 min read
Lessons Learned from Misusing INSERT INTO SELECT in MySQL Data Migration
Zhuanzhuan Tech
Zhuanzhuan Tech
Apr 17, 2024 · Databases

Database Storage Optimization and Migration Strategy for ZhaiZhai Financial System

This article presents a comprehensive analysis of the ZhaiZhai financial system's growing data volume and slow‑query issues, evaluates four storage solutions—including sharding, hot‑cold separation, TiDB, and OceanBase—selects TiDB as the optimal choice, and details the migration steps and Elasticsearch integration to improve performance and scalability.

Data MigrationDatabase OptimizationScalability
0 likes · 13 min read
Database Storage Optimization and Migration Strategy for ZhaiZhai Financial System
Efficient Ops
Efficient Ops
Apr 16, 2024 · Databases

Data Migration & Distributed Transactions: XA, BASE, TCC, AT Guide

This article explores strategies for seamless data migration—including full, incremental, and binlog‑based approaches—and examines distributed transaction models such as XA, BASE, TCC, and AT, outlining their components, workflows, advantages, challenges, and supporting tools like Seata and Canal.

BASEData MigrationDistributed Transactions
0 likes · 9 min read
Data Migration & Distributed Transactions: XA, BASE, TCC, AT Guide
Tencent Cloud Developer
Tencent Cloud Developer
Apr 16, 2024 · Databases

Major Updates and New Features of WCDB 2: Multi‑language Support, SQL Builder, Backup & Recovery, Data Migration, Compression, and Performance Optimizations

WCDB 2 introduces multi-language support (C++, Java, Kotlin), a rewritten SQL builder, fast backup and recovery with sub‑second times, incremental WAL‑based backup, automatic data migration, Zstandard compression, on‑the‑fly column addition, and performance enhancements such as an FTS5 optimizer and interruptible transactions, delivering a robust high‑performance mobile database solution.

Data MigrationSQL BuilderSQLite
0 likes · 47 min read
Major Updates and New Features of WCDB 2: Multi‑language Support, SQL Builder, Backup & Recovery, Data Migration, Compression, and Performance Optimizations
DataFunTalk
DataFunTalk
Apr 9, 2024 · Big Data

Practical Experience and Solutions for Migrating and Optimizing Spark 3.1 in Xiaomi’s One‑Stop Data Development Platform

This article shares Xiaomi's real‑world challenges and solutions when building a new Spark 3.1‑based data platform, covering Multiple Catalog implementation, Hive‑to‑Spark SQL migration, automated batch upgrades, performance and stability optimizations, and future roadmap for vectorized execution.

Apache SparkBig DataData Migration
0 likes · 14 min read
Practical Experience and Solutions for Migrating and Optimizing Spark 3.1 in Xiaomi’s One‑Stop Data Development Platform
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Mar 7, 2024 · Backend Development

Getting Started with Strapi: Installation, Configuration, CRUD Operations, and Data Migration

This guide introduces Strapi, an open‑source headless CMS, covering its key features, quick installation, language switching, basic usage for creating content types, exposing RESTful and GraphQL CRUD APIs, visualizing data with external tools, and migrating SQLite databases to other systems.

Backend DevelopmentCRUDData Migration
0 likes · 8 min read
Getting Started with Strapi: Installation, Configuration, CRUD Operations, and Data Migration
dbaplus Community
dbaplus Community
Feb 20, 2024 · Databases

How to Migrate to AntDB with a Six‑Step Progressive Strategy and Optimize Key Parameters

This article outlines a six‑stage, progressive migration plan for replacing legacy databases with AntDB, covering deployment, extensive testing, SQL refactoring, data migration, go‑live procedures, and monitoring, while providing detailed tuning recommendations for dozens of critical database parameters.

AntDBData MigrationPostgreSQL Parameters
0 likes · 21 min read
How to Migrate to AntDB with a Six‑Step Progressive Strategy and Optimize Key Parameters
dbaplus Community
dbaplus Community
Jan 16, 2024 · Databases

Mastering MariaDB ColumnStore: Installation, Migration, Replication, and Performance Insights

This guide explains MariaDB ColumnStore’s columnar architecture, key advantages, step‑by‑step installation, field and SQL limitations, how to migrate InnoDB data, configure multi‑source replication, compare performance with InnoDB, and achieve high availability and dynamic scaling for analytical workloads.

ColumnstoreData MigrationInstallation
0 likes · 17 min read
Mastering MariaDB ColumnStore: Installation, Migration, Replication, and Performance Insights

Big Data Testing: Methods, Tool Selection, and Practical Implementation with Datacompy

This article introduces big data testing concepts, outlines common testing methods, evaluates the Python library Datacompy against alternatives, and details a practical implementation for large-scale data migration and validation, including configuration, volume comparison, content verification, and performance optimizations such as sorting and multithreading.

Data MigrationPythondata testing
0 likes · 10 min read
Big Data Testing: Methods, Tool Selection, and Practical Implementation with Datacompy
JD Retail Technology
JD Retail Technology
Jan 8, 2024 · Databases

Practical Case Study of System Storage Expansion, Upgrade, and Optimization

This article presents a detailed technical case study on expanding and optimizing a system's storage capacity, covering business background, current architecture, implementation plans, technology selection, data synchronization strategies, phased rollout steps, results, and remaining challenges, with concrete metrics and diagrams.

Data MigrationScalabilitydatabases
0 likes · 11 min read
Practical Case Study of System Storage Expansion, Upgrade, and Optimization
dbaplus Community
dbaplus Community
Nov 2, 2023 · Databases

Scaling a Ride‑Hailing Order System: From Single SQLServer to Sharded MySQL

This article recounts the step‑by‑step evolution of a ride‑hailing order service’s data layer—from a single SQLServer database through SQL optimization, read‑write splitting, business‑domain sharding, caching and MQ integration, to a custom sharding component and smooth migration to MySQL—highlighting challenges, solutions, and lessons learned.

Data MigrationMessage QueueMicroservices
0 likes · 15 min read
Scaling a Ride‑Hailing Order System: From Single SQLServer to Sharded MySQL
Ctrip Technology
Ctrip Technology
Oct 12, 2023 · Backend Development

Evolution and Optimization of Ticket Product Structure at Trip.com

This article examines the evolution of Trip.com’s ticket product structure, detailing its transition from low to high‑level schema, the technical challenges faced across user, merchant, platform and R&D perspectives, and the systematic solutions—including structure integration, sales‑attribute standardization, and description‑attribute layering—that improved efficiency, data quality, and internationalization.

BackendData MigrationSystem Integration
0 likes · 17 min read
Evolution and Optimization of Ticket Product Structure at Trip.com
Huolala Tech
Huolala Tech
Sep 21, 2023 · Big Data

How We Built a Scalable Data Migration Framework for Billions of Transactions

This article details the design and implementation of a custom, high‑throughput data migration framework that handles petabyte‑scale transaction data, supports heterogeneous source/target schemas, ensures zero‑downtime operation, and provides robust scheduling, checkpointing, and fault‑tolerance mechanisms.

Big DataData MigrationDistributed Systems
0 likes · 17 min read
How We Built a Scalable Data Migration Framework for Billions of Transactions
政采云技术
政采云技术
Jul 20, 2023 · Backend Development

Redesigning an Operation Log System: Architecture, Implementation Options, and Historical Data Migration

This article describes the challenges of scaling a multi‑system operation log platform, proposes a new unified log schema, compares non‑intrusive and intrusive collection approaches using Canal and AOP/annotations, and outlines a rule‑engine‑driven migration strategy for legacy log data.

Data MigrationJavaMicroservices
0 likes · 10 min read
Redesigning an Operation Log System: Architecture, Implementation Options, and Historical Data Migration
Code Ape Tech Column
Code Ape Tech Column
Jul 18, 2023 · Backend Development

Comprehensive Guide to Application and Data Splitting Migration for High‑Volume C‑End Services

This article outlines a two‑stage migration strategy—application splitting and data splitting—detailing how to isolate APIs, message queues, and scheduled tasks, manage dual‑write database synchronization, ensure gray‑scale traffic routing, and perform validation to achieve a reliable, low‑risk transition for large‑scale backend systems.

Data MigrationDatabase Syncapplication splitting
0 likes · 12 min read
Comprehensive Guide to Application and Data Splitting Migration for High‑Volume C‑End Services
Open Source Linux
Open Source Linux
Jul 4, 2023 · Operations

Master Redis Monitoring, Migration, and Cluster Management with Prometheus and CacheCloud

This guide walks through essential Redis operations, covering real‑time monitoring with the INFO command and Prometheus‑compatible exporters, data migration using Redis‑shake, consistency verification via Redis‑full‑check, and comprehensive cluster management with CacheCloud, providing practical tools for reliable Redis administration.

Data MigrationOperationsPrometheus
0 likes · 11 min read
Master Redis Monitoring, Migration, and Cluster Management with Prometheus and CacheCloud
Architect's Guide
Architect's Guide
Jun 15, 2023 · Databases

Splitting a Massive MySQL Financial Transaction Table: Challenges, Design, and Implementation

The article describes how a finance team tackled a 50‑million‑row MySQL transaction table by analyzing the pain points, defining split goals, selecting sharding‑jdbc, designing multi‑source transaction handling, pagination across shards, data migration strategies, and a staged rollout to ensure stability and performance.

Backend DevelopmentData MigrationTable Sharding
0 likes · 10 min read
Splitting a Massive MySQL Financial Transaction Table: Challenges, Design, and Implementation
dbaplus Community
dbaplus Community
Jun 11, 2023 · Databases

Splitting a 500‑Million‑Row MySQL Table: Practical Sharding‑JDBC Lessons

Facing a MySQL transaction table exceeding 50 million rows and growing rapidly, a finance team redesigned the schema using sharding‑jdbc, chose transaction time as the shard key, tackled multi‑data‑source transactions and cross‑shard pagination, and executed a phased migration to ensure stability and performance.

Data MigrationLarge TablesSharding-JDBC
0 likes · 11 min read
Splitting a 500‑Million‑Row MySQL Table: Practical Sharding‑JDBC Lessons
iQIYI Technical Product Team
iQIYI Technical Product Team
Jun 9, 2023 · Big Data

Accelerating iQIYI Big Data Platform: Migrating from Hive to Spark SQL

iQIYI accelerated its big‑data platform by migrating the OLAP layer from Hive to Spark SQL, achieving a 67 % speedup, 50 % CPU reduction and 44 % memory savings, while automating the conversion of tens of thousands of tasks and delivering faster analytics for advertising, BI, membership and user‑growth services.

Data MigrationHivePerformance Optimization
0 likes · 18 min read
Accelerating iQIYI Big Data Platform: Migrating from Hive to Spark SQL
Top Architect
Top Architect
May 29, 2023 · Databases

Large Table Splitting in MySQL: Challenges, Solutions, and Lessons Learned

Facing a 50‑million‑row financial transaction table, the team implemented horizontal sharding with sharding‑jdbc, designed migration and pagination strategies, tackled multi‑datasource transaction issues, and outlined a phased rollout, providing practical insights for large‑scale MySQL table splitting and backend system stability.

BackendData MigrationPerformance Optimization
0 likes · 11 min read
Large Table Splitting in MySQL: Challenges, Solutions, and Lessons Learned
Architecture Digest
Architecture Digest
May 16, 2023 · Backend Development

High‑Availability Architecture for a Membership System: Dual‑Center ES Cluster, Redis Caching, MySQL Migration, and Fine‑Grained Flow Control

This article presents a comprehensive engineering case study of a high‑traffic membership system, detailing the dual‑center Elasticsearch high‑availability design, traffic‑isolated three‑cluster ES architecture, Redis caching strategy, dual‑center MySQL partitioning and migration plan, abnormal member relationship governance, and future fine‑grained flow‑control and downgrade policies.

Backend ArchitectureData MigrationElasticsearch
0 likes · 19 min read
High‑Availability Architecture for a Membership System: Dual‑Center ES Cluster, Redis Caching, MySQL Migration, and Fine‑Grained Flow Control