Challenges and Practices in Service‑Oriented Splitting of Qunar Payment System

The second part of the series continues the discussion on Qunar's payment system architecture evolution, focusing on the problems and challenges faced during service‑oriented splitting.

2.2 Service‑oriented splitting challenges – While the new architecture clarifies system structure, it introduces new difficulties in development efficiency, multi‑system management, and daily operations.

2.2.1 Improving development efficiency – The team defines strict conventions for Dubbo services (interface granularity, parameter standards, unified response, business templates, documentation generation) and HTTP services (command definition, validators, parameter types). Example command definition:

<commands>
  <command name="forex_queryExchangeRate">
    <cnName>汇率查询接口</cnName>
    <version>20150808</version>
    <desc>查询本币和目标币种汇率</desc>
    <request>
      <param name="localCurrType" required="true">
        <validator id="CURID"/>
      </param>
      ...
    </request>
    <response>
      <param name="localCurrType">
        <cnName>本币</cnName>
        <required>true</required>
      </param>
      ...
    </response>
  </command>
</commands>

Validators and parameter types are also defined in XML blocks, wrapped similarly in tags.

2.2.2 Managing multiple systems – A unified monitoring template for HTTP and Dubbo services, component‑level monitoring (Redis, Mybatis, Lock, QMQ, EventBus, DataSource, JobScheduler), and automated generation of monitoring dashboards via Python scripts are introduced.

2.3 DB handling during splitting – The article describes table partitioning (time‑based, hash‑based, or hybrid), database sharding with separate instances, and read/write separation with load balancing. Example of a lock annotation for concurrency control:

@RequestLock(
    lockKeyPrefix = "combdaikoupay:",
    lockKey = "${parentMerchantCode}_${parentTradeNo}",
    lockKeyParamMustExists = true,
    lockKeyExpireSecs = 5,
    lockUsedRetryTimes = 0,
    lockUsedRetryLockIntervalMills = 500,
    lockGotExIgnore = false)

Read/write load balancing strategies and dynamic data source selection are illustrated with diagrams.

2.4 Asynchronous processing – Servlet 3 asynchronous support, QMQ, and Dubbo are used to decouple HTTP threads from business processing, improving throughput. A polling‑based flow diagram shows how HTTP services interact with backend transaction systems via QMQ and cache.

2.5 Monitoring and alerting – A Java monitoring module embeds in applications, sending metrics to multiple endpoints; an offline Python monitoring framework collects data from DB, Java APIs, Redis, etc.; a data‑flow system built on XFlume, Kafka, Storm, HDFS, HBase, Redis, Hive processes logs and binlogs; and an alerting system supports mail, SMS, and QTalk with both explicit failure alerts and computed threshold alerts.

3. Summary – After extensive splitting of payment, checkout, gateway, accounting, and core services, the system now handles high‑complexity and high‑concurrency workloads much better, though further improvements are still needed.