Benchmarking TokuDB vs InnoDB: Insertion Throughput and Compression Ratio

Background

TokuDB is a MySQL storage engine that uses a cache‑independent Fractal Tree index structure. It offers high compression (up to 25×), ACID compliance, MVCC, online schema modification, and improved replication performance, making it attractive for log‑storage and analysis workloads.

The test focuses on TokuDB's insert performance and compression ratio, using InnoDB as a reference baseline.

Test Environment

The benchmark runs on a high‑end server with >100 GB RAM, Intel Xeon E5 CPU series, and SSD storage. MySQL version 5.6.28‑76.1 with the TokuDB plugin is installed according to the Percona installation guide.

Engine availability after installation:

+--------------------+---------+--------------+------+------------+
| Engine   | Support | Transactions | XA   | Savepoints |
+--------------------+---------+--------------+------+------------+
| InnoDB   | YES     | YES          | YES  | YES        |
| CSV      | YES     | NO           | NO   | NO         |
| MyISAM   | YES     | NO           | NO   | NO         |
| BLACKHOLE| YES    | NO           | NO   | NO         |
| MEMORY   | YES     | NO           | NO   | NO         |
| TokuDB   | DEFAULT | YES          | YES  | YES        |
| MRG_MYISAM| YES    | NO           | NO   | NO         |
| ARCHIVE  | YES     | NO           | NO   | NO         |
| FEDERATED| NO      | NULL         | NULL | NULL       |
| PERFORMANCE_SCHEMA| YES| NO      | NO   | NO         |
+--------------------+---------+--------------+------+------------+

After confirming the engine list, TokuDB is set as the default storage engine.

Testing Tools and Variables

Sysbench 0.5 (with Lua script support) is chosen as the benchmark tool for its simplicity and flexibility. The mysqldump utility is used to evaluate compression ratios.

Key variables examined:

InnoDB: innodb_flush_log_at_trx_commit and sync_binlog

TokuDB: tokudb_commit_sync and tokudb_fsync_log_period

Compression algorithm selection via tokudb_row_format (values: TOKUDB_QUICKLZ , TOKUDB_ZLIB , TOKUDB_LZMA , TOKUDB_SNAPPY )

Variable meanings (excerpt):

innodb_flush_log_at_trx_commit:
0 – log buffer flushed every second, no sync on commit (fast, risk of 1 s loss)
1 – flush on every commit (safe, slower)
2 – write on commit, flush every second (balanced)

sync_binlog:
0 – no explicit sync (OS decides)
1 – sync after each transaction (safe, slower)

Test Methodology

A classic controlled‑variable approach is used. The dimensions are:

Storage engine (InnoDB vs TokuDB)

Log‑sync mode (synchronous vs asynchronous)

Compression algorithm (four options)

Sysbench thread count ([1,2,4,8,16,24,32,48,64,80])

Number of tables ([1,2,4,8,12,16])

Each combination inserts 2,000,000 rows, and the insert TPS (transactions per second) is recorded.

Results Overview

Four major groups are reported:

InnoDB & Synchronous log flush

InnoDB & Asynchronous log flush

TokuDB & Synchronous log flush

TokuDB & Asynchronous log flush

Key observations:

InnoDB sync mode reaches a peak of ~66 k TPS at 80 threads.

InnoDB async mode peaks around 78 k TPS at 16–32 threads, but overall TPS is lower than the sync case.

TokuDB sync mode peaks at ~60 k TPS (16 threads) and declines after that.

TokuDB async mode shows a similar shape to the sync case; log‑sync has little impact on TokuDB performance.

Representative excerpt from the InnoDB sync table (TPS values):

Table count | 1   | 2   | 4   | 8   | 12  | 16  | 24   | 32   | 48   | 64   | 80
-----------+-----+-----+-----+-----+-----+-----+------+------+------+------+------
1          |5645|10995|18732|33291|54560|66230|66855|56788|55640|58408
2          |...  |...  |...  |...  |...  |...  |...   |...   |...   |...

Similar tables are provided for the other three groups (omitted here for brevity).

Compression Algorithm Comparison

Algorithm

Storage Size

Import Time

snappy

12 GB

47 min 40 s

quicklz

7.1 GB

47 min 21 s

zlib

5.7 GB

48 min 9 s

lzma

4.7 GB

47 min 10 s

Findings:

All algorithms have similar import times.

Compression ratios differ markedly: snappy ≈6.6×, quicklz ≈13×, zlib ≈13.8×, lzma ≈16.8×.

CPU usage: lzma consumes 600‑700 % CPU, while zlib stays between 80‑180 %.

Discussion and Conclusions

InnoDB still leads in raw insert throughput (≈30 % faster).

TokuDB offers 10‑16× storage savings, making it attractive for cold‑data archiving.

Log‑sync mode has minimal impact on TokuDB performance; the default synchronous setting is recommended for data safety.

Among compression algorithms, zlib provides a good balance of compression ratio and CPU overhead and is the recommended default.

TokuDB’s performance potential is not fully realized on multi‑CPU systems, indicating room for future optimization.

Overall, while TokuDB is not yet as mature as InnoDB, its high compression capability and solid insert performance make it a strong candidate for specific workloads such as log storage.