Efficient Data Import with MySQL Shell import_table and Its Performance Compared to LOAD DATA

The author, a DBA familiar with Oracle, MySQL, MongoDB, Redis and TiDB, presents the MySQL Shell import_table tool (full name Parallel Table Import Utility) as a high‑performance alternative to the traditional LOAD DATA statement.

Core capabilities of import_table :

Fully covers all LOAD DATA functions and can be used as a drop‑in replacement.

Supports concurrent import with a default of 8 threads (configurable via the threads parameter).

Allows wildcard file matching to import multiple files at once.

Provides rate limiting through maxRate and custom chunk size via bytesPerChunk .

Handles compressed files, custom character sets, and column‑wise transformations (e.g., upper(@3) , lower(@4) , concat(@3,' ',@4) ).

Basic LOAD DATA example:

load data infile '/data/mysql/3306/tmp/employees_01.csv'
into table employees.emp
character set utf8mb4
fields terminated by ','
enclosed by '"'
lines terminated by '\n'
(@C1,@C2,@C3,@C4,@C5,@C6)
set emp_no=@C1,
    birth_date=@C2,
    first_name=upper(@C3),
    last_name=lower(@C4),
    full_name=concat(first_name,' ',last_name),
    gender=@C5,
    hire_date=@C6,
    modify_date=now(),
    delete_flag=if(hire_date<'1988-01-01','Y','N');

Equivalent import_table call:

util.import_table(
    ['/data/mysql/3306/tmp/employees_01.csv'],
    {
        "schema": "employees",
        "table": "emp",
        "dialect": "csv-unix",
        "skipRows": 0,
        "showProgress": True,
        "characterSet": "utf8mb4",
        "columns": [1,2,3,4,5,6],
        "decodeColumns": {
            "emp_no": "@1",
            "birth_date": "@2",
            "first_name": "upper(@3)",
            "last_name": "lower(@4)",
            "full_name": "concat(@3,' ',@4)",
            "gender": "@5",
            "hire_date": "@6",
            "modify_date": "now()",
            "delete_flag": "if(@6<'1988-01-01','Y','N')"
        }
    })

Advanced usage includes importing multiple files with a wildcard (e.g., "/data/mysql/3306/tmp/employees_*" ), setting threads": "8" for eight concurrent workers, and limiting each thread to maxRate": "2M" (2 MiB/s). The chunk size can be reduced from the default 50 MiB to 1 MiB using bytesPerChunk": "1M" , which lowers transaction size and improves responsiveness.

A performance test importing a 2 GiB sbtest1 table shows LOAD DATA taking about five minutes, while import_table completes in less than half that time, demonstrating a speed‑up of more than 2× on the same hardware.

Technical summary:

import_table implements almost all LOAD DATA features.

It delivers higher import throughput, especially when parallelism and rate‑control options are tuned.

Fine‑grained control over threads, chunk size, and maximum rate helps adapt to different I/O environments.

Detailed progress reporting (speed, elapsed time, batch size, warnings, final summary) simplifies troubleshooting and planning.