BestHub
Sign in
Tagged articles
252 articles
Page 3 of 3
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Jan 7, 2020 · Big Data

Design and Implementation of XFlink: A Flink‑Based Data Migration System on Yarn

The article describes the evolution from the legacy XDATA tool to the new XFlink system, detailing its architecture, core plugins, parser and deployment modules, resource management with Yarn, monitoring via Prometheus and Grafana, and planned enhancements such as Flink SQL configuration and modular plugins.

Big DataData MigrationDistributed Systems
0 likes · 10 min read
Design and Implementation of XFlink: A Flink‑Based Data Migration System on Yarn
DataFunTalk
DataFunTalk
Oct 25, 2019 · Big Data

Migrating Data from HBase to Kafka Using MapReduce

This article explains how to reverse the typical data flow by extracting massive Rowkeys from HBase with MapReduce, storing them on HDFS, and then using batch Get operations to retrieve the full records and write them into Kafka, while handling retries and monitoring progress.

Big DataData MigrationHBase
0 likes · 9 min read
Migrating Data from HBase to Kafka Using MapReduce
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 22, 2019 · Databases

Alibaba Cloud BDS Service for Non‑Stop HBase Cluster Migration

This article explains how Alibaba Cloud's BDS migration service enables continuous, high‑performance migration of HBase clusters—including schema, full data, and incremental sync—across version upgrades, hardware changes, network migrations, and cross‑region scenarios, while ensuring stability and minimal impact on live workloads.

Alibaba CloudBDSBig Data
0 likes · 10 min read
Alibaba Cloud BDS Service for Non‑Stop HBase Cluster Migration
Didi Tech
Didi Tech
Sep 20, 2019 · Big Data

FastLoad: A One-Click DTS Platform for Online Data Migration

FastLoad, Didi’s one‑click DTS platform, accelerates migration of terabyte‑scale offline data into its Fusion storage by using RocksDB’s IngestFile to import SST files directly, cutting a 1 TB load from twelve hours to one, while supporting thousands of daily tasks with 99.99% stability.

DTS platformData MigrationPerformance Optimization
0 likes · 8 min read
FastLoad: A One-Click DTS Platform for Online Data Migration
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 2, 2019 · Databases

Apache Phoenix Tutorial: Quick Start, Data Types, DML, Indexes, Salted Tables, and Advanced Features

This comprehensive guide introduces Apache Phoenix as an HBase SQL layer, covering quick‑start steps, supported data types, DML syntax, salted tables to prevent hotspots, various secondary index types, bulk‑load methods, auto‑increment IDs, dynamic columns, pagination, query plan analysis, and data migration techniques.

Apache PhoenixData MigrationHBase
0 likes · 33 min read
Apache Phoenix Tutorial: Quick Start, Data Types, DML, Indexes, Salted Tables, and Advanced Features
Java Backend Technology
Java Backend Technology
Aug 23, 2019 · Backend Development

How I Cut a 2‑Month Data Migration to 4 Hours: A Backend Performance Journey

This article chronicles a Java backend data‑migration project where the author transformed a two‑month, 2000‑million‑record process into a four‑hour job by iteratively redesigning the architecture from a monolithic procedural approach to a fully decoupled, multithreaded, queue‑driven system, highlighting key performance bottlenecks and solutions.

Data MigrationJavaarchitecture
0 likes · 14 min read
How I Cut a 2‑Month Data Migration to 4 Hours: A Backend Performance Journey
Youku Technology
Youku Technology
Aug 15, 2019 · Big Data

Youku's Migration from Hadoop to Alibaba Cloud MaxCompute: Benefits and Technical Insights

Youku’s 2017 migration from an on‑premises Hadoop cluster to Alibaba Cloud MaxCompute delivered a unified, elastic data pipeline that cut compute and storage costs by roughly half, handled billions of daily log records, boosted performance and scalability, and empowered analysts with self‑service tools and a rich ecosystem.

Big DataCost OptimizationData Migration
0 likes · 12 min read
Youku's Migration from Hadoop to Alibaba Cloud MaxCompute: Benefits and Technical Insights
Programmer DD
Programmer DD
Jul 30, 2019 · Databases

Mastering Database Sharding: When and How to Split Tables and Schemas

This article reviews the complete process of database and table sharding, covering when to shard, range and hash strategies, combining both, data migration challenges, business compatibility, and long‑term archiving to keep MySQL workloads manageable.

Data MigrationHash ShardingRange Sharding
0 likes · 11 min read
Mastering Database Sharding: When and How to Split Tables and Schemas
58 Tech
58 Tech
Jul 25, 2019 · Databases

Design and Evolution of WTable’s Scaling Process Using RocksDB

This article explains how the WTable distributed key‑value store leverages RocksDB’s LSM‑tree architecture and slot‑based data distribution to redesign its scaling workflow, separating full and incremental data migration to reduce compaction overhead and achieve high‑speed, low‑impact cluster expansion.

Data MigrationRocksDBWTable
0 likes · 8 min read
Design and Evolution of WTable’s Scaling Process Using RocksDB
Alibaba Cloud Developer
Alibaba Cloud Developer
Jul 9, 2019 · Big Data

How Youku Cut Costs and Boost Performance by Migrating to MaxCompute

This article explains how Youku processed billions of daily logs, migrated from Hadoop to Alibaba Cloud MaxCompute in 2017, and achieved lower compute and storage costs, faster data delivery, and greater operational flexibility through a robust big‑data platform tailored to its complex business needs.

Cost OptimizationData MigrationData Warehouse
0 likes · 12 min read
How Youku Cut Costs and Boost Performance by Migrating to MaxCompute
Programmer DD
Programmer DD
Jun 16, 2019 · Backend Development

From Two‑Month Crawl to Four‑Hour Sprint: Optimizing a 20M‑Record Data Migration

This article chronicles a step‑by‑step performance overhaul of a 20‑million‑record migration project, detailing four architectural revisions—from a single‑threaded procedural script to a fully decoupled, multithreaded, interface‑driven solution—that reduced processing time from two months to just four hours while ensuring data consistency, recoverability, and scalability.

Batch ProcessingData MigrationJava
0 likes · 15 min read
From Two‑Month Crawl to Four‑Hour Sprint: Optimizing a 20M‑Record Data Migration
Aikesheng Open Source Community
Aikesheng Open Source Community
May 30, 2019 · Databases

Oracle GoldenGate Migration Guide: Oracle to MySQL Data Replication

This article provides a comprehensive step‑by‑step guide for migrating data from Oracle to MySQL using Oracle GoldenGate, covering architecture overview, environment setup, table and data migration, configuration of source and target processes, handling of common issues, data validation, and reference resources.

Data MigrationDatabase ReplicationOGG
0 likes · 22 min read
Oracle GoldenGate Migration Guide: Oracle to MySQL Data Replication
Qunar Tech Salon
Qunar Tech Salon
May 16, 2019 · Big Data

Optimizing HDFS Federation Data Migration with FastCopy and qFastCopy at Qunar

This article describes the challenges of scaling Qunar's Hadoop NameNode, introduces HDFS Federation and the FastCopy tool, presents performance tests comparing FastCopy with DistCp, and details the development and evaluation of an optimized qFastCopy solution that reduces multi‑petabyte migration time from hours to a few.

Big DataData MigrationFastCopy
0 likes · 8 min read
Optimizing HDFS Federation Data Migration with FastCopy and qFastCopy at Qunar
ITPUB
ITPUB
May 14, 2019 · Databases

How Didi Built Fusion: From NoSQL to NewSQL and Distributed Database Design

This article details Didi's evolution from a self‑built NoSQL system called Fusion to a NewSQL solution and outlines the challenges, architectural highlights, data migration mechanisms, and future distributed database design, providing concrete metrics, diagrams, and practical lessons learned.

Data MigrationNewSQLNoSQL
0 likes · 18 min read
How Didi Built Fusion: From NoSQL to NewSQL and Distributed Database Design
Architects' Tech Alliance
Architects' Tech Alliance
Apr 16, 2019 · Cloud Computing

How OpenSDS Gelato Enables Unified Multi‑Cloud Object Storage and Data Migration

OpenSDS Gelato is an open‑source multi‑cloud data management platform that unifies object storage across public clouds and on‑premises, offers S3‑compatible APIs, supports backend pooling and policy‑driven cross‑cloud data flow, and outlines a roadmap for lifecycle management and advanced migration tools.

Data MigrationOpenSDSS3 Interface
0 likes · 7 min read
How OpenSDS Gelato Enables Unified Multi‑Cloud Object Storage and Data Migration
dbaplus Community
dbaplus Community
Mar 12, 2019 · Databases

Mastering HBase Cross‑Datacenter Migration: Snapshots, Architecture, and Real‑World Tips

This article provides a comprehensive technical guide on HBase, covering its core concepts, advantages and drawbacks, architecture layers, practical use cases, and a detailed step‑by‑step process for large‑scale cross‑datacenter migration using snapshot‑based strategies, with commands, diagrams, and lessons learned.

Big DataData MigrationDatabase Architecture
0 likes · 19 min read
Mastering HBase Cross‑Datacenter Migration: Snapshots, Architecture, and Real‑World Tips
Tencent Cloud Developer
Tencent Cloud Developer
Nov 5, 2018 · Databases

Best Practices for Redis Database on the Cloud

In this talk, Tencent Cloud senior engineer Zou Peng outlines cloud‑native best practices for Redis 4.0 clusters, detailing a smart‑proxy architecture with gossip‑based management, lazy‑scan big‑key handling, automatic fail‑over, the high‑performance CKV engine, and practical guidance on scaling, sharding, monitoring, and cost‑effective high‑availability deployment.

ClusterData MigrationProxy
0 likes · 14 min read
Best Practices for Redis Database on the Cloud
DataFunTalk
DataFunTalk
Oct 19, 2018 · Databases

HBase Application and High‑Availability Practices

This article summarizes the current usage of HBase at Ping An Technology, the challenges it addresses, detailed client‑ and server‑side performance and stability optimizations, high‑availability mechanisms, data migration strategies, monitoring and repair practices, and future development plans.

Data MigrationHBasePerformance Optimization
0 likes · 9 min read
HBase Application and High‑Availability Practices
iQIYI Technical Product Team
iQIYI Technical Product Team
Sep 21, 2018 · Databases

Case Study of iQIYI’s Adoption of TiDB for Scalable High‑Availability Database Services

iQIYI migrated its critical Edge Control, Video Transcoding, and User Login services from MySQL to TiDB, gaining automatic sharding, high‑availability multi‑datacenter replication, and stable query performance that eliminated storage bottlenecks, complex sharding logic and frequent downtime, while enabling future OLTP/OLAP integration.

Data MigrationScalabilityTiDB
0 likes · 10 min read
Case Study of iQIYI’s Adoption of TiDB for Scalable High‑Availability Database Services
Java Backend Technology
Java Backend Technology
Jul 29, 2018 · Backend Development

Turning a 2‑Month Data Migration into 4 Hours: Backend Performance Secrets

After struggling with a two‑month data migration that processed 20 million users, the author iteratively redesigned the system—from a single‑threaded, procedural approach to a fully decoupled, multithreaded architecture using queues and batch operations—ultimately reducing runtime to just four hours while ensuring data consistency and recoverability.

Batch ProcessingData Migrationmultithreading
0 likes · 13 min read
Turning a 2‑Month Data Migration into 4 Hours: Backend Performance Secrets
Hujiang Technology
Hujiang Technology
Jul 5, 2018 · Databases

Order Table Splitting Project: Architecture, Implementation, and Performance Improvements

This article describes the background, analysis, solution selection, architecture redesign, execution steps, and results of a large‑scale order table splitting project that separates hot and cold data, applies horizontal sharding, and uses MySQL partitioning to control table size and boost query performance.

Backend DevelopmentData Migrationdatabase partitioning
0 likes · 15 min read
Order Table Splitting Project: Architecture, Implementation, and Performance Improvements
Alibaba Cloud Developer
Alibaba Cloud Developer
Apr 19, 2018 · Databases

How HiTSDB’s New Streaming Aggregation Engine Boosts Query Speed 10×

This article examines the architectural redesign of Alibaba's High‑Performance Time Series Database (HiTSDB), covering storage model changes, inverted‑index enhancements, a pipelined streaming aggregation engine, data‑migration strategies, and performance benchmarks that together deliver over tenfold query speed improvements.

Data MigrationHiTSDBPerformance Optimization
0 likes · 24 min read
How HiTSDB’s New Streaming Aggregation Engine Boosts Query Speed 10×
dbaplus Community
dbaplus Community
Dec 12, 2017 · Databases

MariaDB ColumnStore: Future‑Ready OLAP Architecture, Limits & Migration Tips

This article shares practical experience deploying MariaDB ColumnStore for offline OLAP, covering its MPP‑based architecture, user‑experience benefits, high‑availability options, numerous syntax and data‑type limitations, and detailed scripts for schema conversion and incremental data extraction using pt‑archiver.

ColumnstoreData MigrationMariaDB
0 likes · 17 min read
MariaDB ColumnStore: Future‑Ready OLAP Architecture, Limits & Migration Tips
Suning Technology
Suning Technology
Nov 13, 2017 · Backend Development

How Suning Scaled Its Membership System for Double‑11: From Legacy POS to Multi‑Active Architecture

This article examines Suning's evolution of its membership platform—from an early offline POS system to a vertically split, cloud‑native architecture—detailing capacity planning, performance testing, data migration with Spark, multi‑active deployment, and future plans for cross‑region high availability.

Big DataCloud NativeData Migration
0 likes · 15 min read
How Suning Scaled Its Membership System for Double‑11: From Legacy POS to Multi‑Active Architecture
ITPUB
ITPUB
Nov 12, 2017 · Backend Development

How Suning Scaled Its Membership System for Double‑11: Architecture Evolution and Multi‑Active Deployment

The article details Suning’s decade‑long evolution of its membership platform for the Double‑11 shopping festival, covering early offline CS architecture, the transition to a WebSphere‑based e‑commerce system, vertical module splitting, data migration with Spark and Hive, and the implementation of same‑city multi‑active deployment.

BackendData MigrationScalability
0 likes · 18 min read
How Suning Scaled Its Membership System for Double‑11: Architecture Evolution and Multi‑Active Deployment
Architects' Tech Alliance
Architects' Tech Alliance
Jul 1, 2017 · Operations

Comprehensive Overview of Data Migration: Purposes, Challenges, and Main Technologies

This article provides a detailed introduction to data migration, explaining why and when it is needed, outlining its major challenges such as security, compatibility and downtime, and describing the four primary migration approaches—application, volume‑manager, network, and storage—along with common tools and technologies.

Data MigrationLVMVirtualization
0 likes · 10 min read
Comprehensive Overview of Data Migration: Purposes, Challenges, and Main Technologies
ITPUB
ITPUB
Mar 21, 2017 · Databases

How to Fix Chinese Character Garbling When Migrating Oracle to MySQL

When moving data from Oracle to MySQL, Chinese characters may appear as garbled text because the server uses a latin1 charset; this guide explains three levels of fixes—table, database, and server—by configuring UTF‑8 settings and verifying the changes.

Character SetChineseData Migration
0 likes · 5 min read
How to Fix Chinese Character Garbling When Migrating Oracle to MySQL
Efficient Ops
Efficient Ops
Mar 7, 2017 · Big Data

How Tencent Scaled Its TDW to 8,800 Nodes and Mastered Cross-City Data Migration

Tencent’s senior engineer explains how the TDW (Tencent Distributed Data Warehouse) grew from a few hundred to thousands of nodes, the challenges of cross‑city migration, and the modeling, relationship‑chain, dual‑write tables, and platform strategies they built to ensure seamless, low‑impact data and task migration.

Big DataData MigrationTDW
0 likes · 26 min read
How Tencent Scaled Its TDW to 8,800 Nodes and Mastered Cross-City Data Migration
Practical DevOps Architecture
Practical DevOps Architecture
Nov 22, 2016 · Databases

Common MySQL SQL Statements and Table Alteration Cheat Sheet

This article compiles frequently used MySQL SQL commands—including data migration, multi‑table updates, conditional updates, aggregation inserts, and various ALTER TABLE operations such as adding indexes, dropping and adding columns—to serve as a practical reference for database developers.

DDLDMLData Migration
0 likes · 6 min read
Common MySQL SQL Statements and Table Alteration Cheat Sheet
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Nov 13, 2016 · Databases

How to Scale an Order System Beyond 200GB with Vertical & Horizontal Sharding

This article explains how a massive order database exceeding 200 GB was vertically and horizontally sharded, introduces query, range, and hash sharding strategies, presents a custom timestamp‑based unique ID scheme, and outlines a multi‑phase data migration plan to achieve long‑term scalability.

Data Migrationhorizontal-splittingorder system scaling
0 likes · 9 min read
How to Scale an Order System Beyond 200GB with Vertical & Horizontal Sharding
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Oct 13, 2016 · Databases

How to Scale Order Systems with Horizontal Database Sharding: A Real‑World Case Study

This article presents a comprehensive, practice‑driven analysis of horizontal database sharding for a high‑traffic e‑commerce order system, covering sharding dimensions, strategies, quantity planning, transparent routing, pagination challenges, lookup mapping, overall architecture, deployment steps, and the measurable performance and cost benefits achieved.

Data MigrationOracledatabase scaling
0 likes · 14 min read
How to Scale Order Systems with Horizontal Database Sharding: A Real‑World Case Study
ITPUB
ITPUB
Sep 7, 2016 · Databases

Optimizing and Migrating a 20 Million‑Row MySQL Table

This guide explains how to clean redundant fields, restructure a massive MySQL table, batch‑fetch data with LIMIT, rewrite slow queries, and compare three insertion methods—including prepared statements and bulk inserts—to cut migration time from hours to under 20 minutes.

Batch ProcessingData MigrationSQL Optimization
0 likes · 7 min read
Optimizing and Migrating a 20 Million‑Row MySQL Table
Meituan Technology Team
Meituan Technology Team
Sep 2, 2016 · Databases

Automated Database Operations Platform Overview

An automated database operations platform shifts routine DBA tasks to developer self‑service and fully automates processes such as cluster provisioning, scaling, backup, migration, and sharding, using a stateless workflow center, job queue, and unified SqlEditor to improve efficiency, safety, and auditability.

AutomationData MigrationJob Center
0 likes · 8 min read
Automated Database Operations Platform Overview
dbaplus Community
dbaplus Community
Jul 17, 2016 · Databases

How JD Scaled Its One‑Yuan Grab Treasure System with Database Sharding and ES Aggregation

This article details JD's One‑Yuan Grab Treasure platform redesign, covering business growth drivers, database sharding estimation, hash‑plus‑range routing implementation, Elasticsearch aggregation, Canal‑based sync, historical data migration, and downgrade mechanisms to ensure high‑throughput, reliable order processing during massive sales events.

Backend ArchitectureData MigrationPerformance Scaling
0 likes · 11 min read
How JD Scaled Its One‑Yuan Grab Treasure System with Database Sharding and ES Aggregation
Architects' Tech Alliance
Architects' Tech Alliance
Jul 1, 2016 · Operations

Understanding Data Migration: Challenges, Techniques, and Best Practices

This article explains why data migration is essential for maintaining secure and reliable data services, outlines the typical triggers such as equipment end‑of‑life, and details the major challenges—including security, compatibility, downtime, and required expertise—while reviewing the four primary migration approaches and their practical implementations.

Data MigrationSystem Integration
0 likes · 9 min read
Understanding Data Migration: Challenges, Techniques, and Best Practices
ITPUB
ITPUB
May 20, 2016 · Databases

Step-by-Step Guide: Migrating DB2 v9.7 PHSDB to K-DB11G

This tutorial details the complete process of moving a single‑node DB2 v9.7 PHSDB database to K‑DB11G, covering schema‑to‑user mapping, object conversion, constraint handling, identity column emulation, manual adjustments for views/functions, and the export‑import scripts required for data migration.

DB2Data MigrationK-DB
0 likes · 10 min read
Step-by-Step Guide: Migrating DB2 v9.7 PHSDB to K-DB11G
Architecture Digest
Architecture Digest
Apr 27, 2016 · Databases

Database Sharding Strategies: Vertical and Horizontal Partitioning, ID Generation, and Migration Phases

The article outlines a comprehensive database sharding solution for a massive order system, describing vertical and horizontal partitioning, various sharding strategies, a custom ID scheme, transaction handling, complex query constraints, and a three‑stage data migration plan to achieve long‑term scalability.

Data MigrationID generationScalability
0 likes · 10 min read
Database Sharding Strategies: Vertical and Horizontal Partitioning, ID Generation, and Migration Phases
dbaplus Community
dbaplus Community
Apr 13, 2016 · Databases

Secure Redis Cluster: Adding Password Authentication and Automated Node Management

This guide explains why the official Redis Cluster tools lack password support, outlines the security risks of an unauthenticated cluster, and introduces a custom management utility that adds password authentication, automates slot migration, and simplifies adding or removing nodes, complete with step‑by‑step testing procedures.

Cluster ManagementData MigrationRedis Cluster
0 likes · 8 min read
Secure Redis Cluster: Adding Password Authentication and Automated Node Management
Architect
Architect
Jul 8, 2015 · Databases

MySQL Architecture Design: Business Analysis, Common Deployment Patterns, Sharding Strategies, and High‑Availability Practices

This article explains MySQL architecture design for 24/7 online services, covering business analysis of read‑write patterns, common deployment models, consistency levels, data migration, sharding strategies (vertical, horizontal, and business‑level), and cluster scaling for high availability.

Data MigrationDatabase Architecturemysql
0 likes · 17 min read
MySQL Architecture Design: Business Analysis, Common Deployment Patterns, Sharding Strategies, and High‑Availability Practices
MaGe Linux Operations
MaGe Linux Operations
Jul 9, 2014 · Databases

SQL Server vs Oracle: 45 Essential Function Equivalents

This article presents a comprehensive side‑by‑side mapping of 45 common SQL Server functions to their Oracle equivalents, covering numeric, string, date, and miscellaneous operations with example queries and notes on usage differences.

Data MigrationDatabase FunctionsOracle
0 likes · 16 min read
SQL Server vs Oracle: 45 Essential Function Equivalents