ElasticSearch Dual‑Data‑Center Architecture and Implementation at Xinwang Bank

Introduction

Xinwang Bank, the first internet bank in western China, has pursued a data‑driven development model since its launch, accumulating massive amounts of data. Early on, data was stored in an HBase cluster, but as volume grew, query latency became unacceptable, prompting the adoption of ElasticSearch as a high‑performance, near‑real‑time search engine to replace HBase in core business lines.

Technical Solution

Given the stringent continuity requirements of online banking services, a dual‑data‑center architecture was required for ElasticSearch to ensure high availability and disaster recovery. Common approaches such as single‑cluster cross‑datacenter deployment, dual‑write, or the Platinum‑level Cross‑Cluster Replication (CCR) were evaluated; each had drawbacks in latency, consistency, or cost. The chosen solution parses ElasticSearch translog files to achieve real‑time data replication while shielding applications from the underlying data movement.

Table 1. Comparison of ElasticSearch dual‑data‑center construction schemes

ElasticSearch operates in a peer‑to‑peer distributed mode, causing frequent inter‑node communication. Deploying a single cluster across data centers would increase latency and maintenance complexity, so a multi‑cluster, multi‑site deployment was adopted.

The translog‑based approach was selected because it guarantees real‑time synchronization, abstracts data handling from applications, and enables read‑write separation.

Technical Construction

1. Translog File Overview

The translog is ElasticSearch’s transaction log, recording all shard operations (add/update/delete). Each shard has two associated files: translog‑N.tlog (the log file, where N is the generation) and translog‑N.ckp (metadata containing offset, operation size, and generation). Files without a generation number indicate ongoing writes.

2. Translog Parsing

The parsing strategy follows a “partial forward, complete catch‑up” model: after skipping the previously read offset, the parser reads new data, then waits for the current log file to close before performing a full read to capture any missed entries, ensuring both completeness and timeliness.

Figure 1. Translog parsing method for a shard

3. Online Deployment

The solution is packaged as a standalone application named X‑CCR, deployed on ElasticSearch node servers. X‑CCR handles bi‑directional data synchronization and provides read‑write and hot‑cold query separation. Deployment details are shown in Figure 2.

Figure 2. Dual‑data‑center deployment architecture

Performance Evaluation

Xinwang Bank operates two ElasticSearch data centers, each with three physical machines. Under a write load of 50,000 TPS on primary shards, the X‑CCR tool ensures that 75% of the data becomes visible within 2 seconds, meeting the bank’s latency requirements. Detailed metrics are illustrated in Figure 3.

Figure 3. Performance test results of the X‑CCR synchronization tool

Conclusion

The case study demonstrates that Xinwang Bank’s dual‑data‑center ElasticSearch implementation successfully satisfies functional and performance demands, with the first version now operational. Future work includes integrating the solution with ElasticSearch plugins to simplify deployment and management.