AI Automates OSS Domain Migration Across 100M+ Records in 5 Hours

Background

The company needed to urgently unify OSS access domains across all software systems. Original file URLs (e.g., https://domain1.com/image.png) had to be replaced with new domains ( https://domain2.com/image.png) before the old domains went offline, otherwise users would lose access to files and images.

Solution Approaches

Two typical solutions exist: (1) find and replace every stored URL directly in the databases, or (2) add a generic URL translation API that converts old domains to new ones on the fly. If data volume is small, approach 1 suffices; for massive, scattered data, approach 2 is better. However, both require first identifying all file/image URLs in the system.

Challenge: Discovering All URLs

The team owned 10+ databases, 1,000+ business tables, and over 100 million records containing URL-like data. Manual table-by-table inspection, adaptation, and testing was unrealistic within the tight deadline.

AI-Assisted Process

The author decided to try an LLM (Doubao) to automate the workflow, which took roughly 5 hours end-to-end. The process was split into four steps.

Step 1: Collect Data

Initially, the author manually wrote SQL to extract distinct domains from a few key tables, e.g.:

SELECT tt.domain
FROM (
  SELECT SUBSTRING_INDEX(image_url, '/', 3) AS domain
  FROM tc2_customer
  WHERE image_url IS NOT NULL
  AND image_url <> ''
) tt
GROUP BY tt.domain

This returned domains like https://domainA.com, https://domainB.com. But with hundreds of tables, manual querying was infeasible. The author then exported the full database schema and asked the LLM to analyze field names and comments to identify all potential file/image/document URL columns, then generate the complete aggregation SQL for each table. The LLM succeeded, and the generated SQL was executed directly; results were saved to temporary files.

Step 2: Clean Data

Collected results from multiple databases contained many duplicate domains. The author fed all collected domains to the LLM with a simple prompt:

请对以上信息做一个汇总并移除重复内容，输出一份可复制的文件

The LLM deduplicated the list, producing a clean domain inventory.

Step 3: Complete Data

The clean domain list still needed validation via the translation API. The author provided the list and the API specification to the LLM, asking it to call the API for each domain and compile the mappings. After a few iterations, a complete mapping table was produced:

原始域名 → 转换后域名
域名A → 新域名A
域名B → 新域名B
域名C → 新域名C

Step 4: Output Report

Finally, the mapping data was handed to the LLM with a prompt to generate an Excel file:

请将下面的 markdown 的表格数据，转成 excel，要求：必须根据下面的内容生成，excel 内容为文本格式，禁止胡编乱造、甚至修改数据

The LLM produced a downloadable Excel file. The author opened it to verify all mappings; any incorrect conversions were re-configured and re-validated until everything matched expectations.

Conclusion

When a task is well-defined, AI efficiency is remarkable—especially for large-scale, rule-based, repetitive work. The 10+ databases, 1,000+ tables, and 100M+ records would have taken a team days just for data collection; with AI, the whole loop closed in ~5 hours. The key is not to offload everything to AI, but to decompose the problem clearly, instruct AI precisely, and rigorously validate its output. AI shifts the human role from manual execution to thinking, judging, and quality control.