Migrating Redis db8 to db15 Using a Custom Python Script

1 Business Background

Due to business requirements, the Redis cluster's db8 needs to be migrated to another server's db15 without moving the entire cluster.

The situation is illustrated in the diagram below.

Various open‑source tools were explored but none produced satisfactory results, so a custom script was written.

2 Script Section

The following is part of the migrate_redis.py script.

import redis
import time

# 定义 redis1 和 redis 主机信息
redis1_host = '192.168.1.1'
redis1_port = 6579
redis1_db = 8
redis1_password = 'xxxx'

redis2_host = '192.168.1.1'
redis2_port = 6579
redis2_db = 15
redis2_password = 'xxxx'

# 连接 redis1 和 redis2 并验证密码
redis1 = redis.StrictRedis(host=redis1_host, port=redis1_port, db=redis1_db, password=redis1_password)
redis2 = redis.StrictRedis(host=redis2_host, port=redis2_port, db=redis2_db, password=redis2_password)

# 设置每次批量迁移的数据量
batch_size = 1000

# 为进度跟踪初始化变量
keys_processed = 0
start_time = time.time()

# 使用 SCAN 批量获取 key
cursor = '0'
total_keys = len(redis1.keys('*'))

while cursor != 0:
    cursor, keys = redis1.scan(cursor, count=batch_size)
    for key in keys:
        key_data = redis1.dump(key)
        redis2.restore(key, 0, key_data, replace=True)
        keys_processed += 1
        # 每 1000 个 key 打印一次进度
        if keys_processed % batch_size == 0 or keys_processed == total_keys:
            elapsed_time = time.time() - start_time
            keys_per_second = batch_size / elapsed_time
            estimated_remaining_time = (total_keys - keys_processed) / keys_per_second
            print(f"Processed {keys_processed}/{total_keys} keys. "
                  f"Elapsed Time: {elapsed_time:.2f} seconds. "
                  f"Estimated Remaining Time: {estimated_remaining_time:.2f} seconds for the next 1000 keys.")
            # 为下一批次重置变量
            start_time = time.time()

print("Data migration completed.")

3 Output Result

The script prints progress every 1000 keys and estimates the remaining migration time.

Processed 1000/3592 keys. Elapsed Time: 16.46 seconds. Estimated Remaining Time: 42.67 seconds for the next 1000 keys.
Processed 2000/3592 keys. Elapsed Time: 16.96 seconds. Estimated Remaining Time: 27.01 seconds for the next 1000 keys.
Processed 3000/3592 keys. Elapsed Time: 17.03 seconds. Estimated Remaining Time: 10.08 seconds for the next 1000 keys.
Processed 3592/3592 keys. Elapsed Time: 9.81 seconds. Estimated Remaining Time: 0.00 seconds for the next 1000 keys.
Data migration completed.
Process finished with exit code 0

4 Post‑Migration Check

Run the info command to verify: the source contains 3592 keys, the target also contains 3592 keys, confirming successful migration.