Financial Industry Distributed Database Requirements and Selection – Q&A Summary

On June 10, Wei Yadong, manager of the Industrial and Commercial Bank of China (ICBC) Software Development Center, delivered an online live session titled "Financial Industry Distributed Database Requirements and Selection" and answered audience questions, resulting in 31 Q&A pairs.

Q 1: Why not consider the MGR architecture?

A: The MGR architecture was very immature in 2017, with many bugs, and was not suitable for production.

Q 2: Why choose DBLE?

A: Distributed access layers are complex; after evaluating DDB, TDDL, MySQL Proxy, MySQL Router, and DBLE, only DBLE was usable and offered many optimizations over MyCat. However, its configuration is cumbersome, so the team performed deep development and built a unified configuration center.

Q 3: What does the underlying storage look like and where does distribution manifest?

A: The foundation is IAAS with SSD persistence. Distribution appears in two ways: (1) lightweight distributed databases enabling elastic scaling, and (2) sharded storage with a distributed data access layer.

Q 4: Does the original table still exist when using DBLE middleware?

A: No; sharded tables share the same name.

Q 5: Does DBLE support distributed transactions? How are they handled?

A: DBLE does not support distributed transactions directly; they are handled via the bank's distributed transaction framework, XA (2PC/3PC) for strong consistency, or by splitting transactions into primary and secondary parts processed via a distributed message queue.

Q 6: Is the architecture 128 shards with one master and three replicas per shard?

A: Typically four shards are deployed together, with a current setup of 1 master and 4 backups (3 semi‑sync, 1 async remote).

Q 7: What factors influence the sharding rule for the 128 shards?

A: Sharding algorithms are chosen based on business rules, such as using ID card numbers, card numbers, region codes, etc., via modulo or consistent hashing.

Q 8: Are there applications that access all 128 shards at once?

A: No; such access is prohibited by standards.

Q 9: What issues arise with MySQL containerization and can performance handle them?

A: Issues include container IP drift and I/O bottlenecks; solutions involve using SSDs and managing sharding.

Q 10: How is MySQL container data persistence achieved?

A: Persistence is provided by IAAS with SSD storage.

Q 11: What network and storage solutions are used on IAAS?

A: FC‑SAN and SSD.

Q 12: How is a 10 ms low latency achieved?

A: High‑performance hardware, high‑speed networking, and splitting transactions into small units.

Q 13: Are TCC transactions required to be strongly consistent?

A: It depends on the business; some use primary‑secondary transactions, others use XA for strong consistency, while many adopt SAGA for high concurrency scenarios.

Q 14: Are MySQL instances created via operators?

A: ICBC uses K8s, SDN, and IAAS to build stateful container clusters, employing operators for binding resources and fixed IPs.

Q 15: How is semi‑sync degradation addressed?

A: By reducing transaction size (≈10 k rows per commit) and ensuring each table has an auto‑increment primary key.

Q 16: How does semi‑sync guarantee RPO‑0?

A: Similar to Q15, plus monitoring master‑slave consistency and using distributed caches with compensation on failover.

Q 17: Are cross‑node join queries supported?

A: Not recommended; if needed, use Spark or Flink for processing.

Q 18: How are sharding keys chosen?

A: Defined per application, e.g., card BIN for credit cards, ID suffixes for personnel, etc.

Q 19: How to handle complex joins across more than three tables?

A: Ensure proper normalization (3NF/BCNF) or use denormalization to trade space for time.

Q 20: Describe the four types of distributed transaction handling used by ICBC.

A: Low‑concurrency accounting uses XA; high‑concurrency uses a distributed transaction framework (TCC or SAGA), with SAGA currently favored.

Q 21: How does ICBC manage its data models?

A: A proprietary system manages data models and integrates with pipelines for automated script versioning.

Q 22: How to prevent batch processing from exhausting connections needed for online transactions?

A: Batch and online workloads are isolated by policy.

Q 23: Does multi‑center active‑active deployment cause cross‑WAN access issues?

A: No; such latency would be unacceptable.

Q 24: What is ICBC's MySQL high‑availability architecture?

A: 1 master with 4 backups (3 semi‑sync, 1 async remote); early disaster recovery used disk replication with long RTO.

Q 25: In active‑active setups, are applications connected to the same or different databases?

A: A single virtual IP (VIP) via DNS points to the current master, decoupling application configuration from specific nodes.

Q 26: What pitfalls exist when migrating from Oracle to MySQL?

A: Challenges include multi‑table joins, mandatory sharding, device operations, converting stored procedures to Java, transaction control, and SQL tuning.

Q 27: How is Oracle sequence migrated to MySQL?

A: ICBC implements its own sequence service; alternatives include Snowflake algorithm.

Q 28: Is the MySQL version used by ICBC too low?

A: As of late 2016, 5.7 was the best stable version; 8.0 was immature.

Q 29: Differences between MariaDB and MySQL?

A: Versions differ and incompatibilities are common; choose based on requirements.

Q 30: How does ICBC use Redis for distributed caching?

A: Redis is the primary distributed cache, replacing the earlier SSDB solution.

Q 31: What is the presenter’s background?

A: He is a technical and security manager with development roots, holding Oracle and MySQL OCP certifications, and emphasizes broad skill development.

For the full live session recording, see the original article: 《工行数据库选型与分布式架构设计》 .