JDTX Distributed Transaction Solution: Design Principles, Features, and Implementation Challenges

In the era of distributed databases, NewSQL, and cloud‑native databases, incremental enhancements are preferred over radical changes; transparent data sharding and horizontal partitioning are core concepts for handling massive data, with elasticity, HTAP, and distributed transaction middleware like JDTX forming the backbone of modern database solutions.

JDTX, developed by JD Digits' data R&D team, is a distributed transaction middleware that aims to provide strong consistency (full ACID support), high performance surpassing local transactions, and a one‑phase commit (1PC) model, currently targeting relational databases while offering an open SPI for NoSQL integration.

The article outlines the background of database transactions, distinguishing local transactions from distributed ones, and reviews traditional two‑phase commit (XA) protocols, highlighting their suitability for short, low‑latency transactions but limitations in high‑concurrency scenarios.

It then introduces flexible (BASE) transactions, which relax strict consistency and isolation in favor of higher throughput, citing Saga and TCC as typical implementations.

JDTX’s architecture consists of a Transaction Manager (TM) that generates globally ordered log sequence numbers and handles commit/rollback, and a Resource Manager (RM) that separates active (in‑memory MVCC) data from persisted (WAL) data, merging them during queries based on isolation levels.

Key advantages include lossless transaction support via WAL + MVCC, atomicity and consistency through a single‑node MVCC cache, support for multiple isolation levels (read‑committed, repeatable‑read), durability via WAL recovery, high performance from asynchronous flushing, no UNDO log rollback, high availability through sharding and master‑slave replication, and cross‑database transaction capability across MySQL, PostgreSQL, MongoDB, Redis, etc.

Implementation challenges are detailed: building a custom MVCC engine (using xmin/xmax markers similar to PostgreSQL), handling MVCC cleanup (vacuum) especially for long‑running transactions, and integrating SQL query processing by leveraging ShardingSphere’s SQL parser to merge active MVCC data with persisted data for both OLTP (SPJ) and OLAP queries.

The article also lists practical limitations of JDTX, such as the necessity to access databases through JDTX, ongoing SQL compatibility work, and inability to handle tables without primary keys.

Integration with Apache ShardingSphere is described, where ShardingSphere provides JDBC, MySQL, and PostgreSQL proxy entry points, and its SPI allows JDTX to plug in seamlessly, enabling combined data sharding and distributed transaction capabilities.

Future plans for JDTX include improving SQL compatibility, supporting additional databases, implementing SSI isolation level, enhancing management and monitoring tools, deeper integration with Kubernetes and cloud‑native platforms, and positioning JDTX as a standard solution for distributed transactions.