Performance Testing and Optimization of ClickHouse and Elasticsearch for High-Concurrency Scenarios

1 Requirement Analysis

ClickHouse is a column‑oriented DBMS that stores data by columns and executes queries in a vectorized manner, reducing processing overhead. Elasticsearch is an open‑source distributed search and analytics engine built on Apache Lucene, providing real‑time document storage, search and analytics capabilities.

Both systems need stress testing to verify their ability to handle complex, high‑volume queries during peak business periods such as Double‑11, and to locate performance bottlenecks for targeted optimization.

1.2 Test Objectives

The goal is to ensure that ClickHouse and Elasticsearch can sustain the expected load, identify bottlenecks, and improve system performance before the promotion peak.

2 Test Environment Preparation

The test environment mirrors the production environment, using identical hardware and software configurations.

3 Monitoring Tools

http://origin.jd.com/ – JVM and method‑level monitoring.

http://console.jex.jd.com/ – Exception stack, flame graph, thread analysis.

http://x.devops.jdcloud.com/ – CPU usage of ClickHouse/Elasticsearch nodes.

http://dashboard.fireeye.jdl.cn/ – Application server CPU and memory.

4 Load Test Execution and Result Analysis

4.1 Load Test Script

Forcebot (http://forcebot.jd.com) is used to write scripts, configure monitoring, set scenarios, run tasks and collect reports.

4.2 Test Data Design

Key terms: DBCP (database connection pool), maxTotal, max_thread, coordinating nodes, data nodes.

Test data includes ClickHouse nodes (32 C 128 G × 2 replicas), Elasticsearch nodes (same), application servers (4 CPU 8 GB × 2), and maxTotal=16.

4.3 Result Analysis

ClickHouse supports concurrency; increasing max_thread improves TPS. Example: max_thread=32 yields TPS 37, TP99 122; max_thread=1 yields TPS 86, TP99 206.

Elasticsearch shows higher TPS (192) but much larger latency (TP99 3050 ms).

Conclusion: ClickHouse meets business requirements.

Optimization Recommendations

Scale out Elasticsearch coordinating nodes.

Increase big‑data application thread pool to 200 and maxTotal to 50.

Add memory cache to configuration file utilities.

Code example for the queryOBBacklogDataCKNew method:

/**
 * 切ck-queryOBBacklogData
 * @param queryBO
 * @return
 */
public OutboundBacklogRespBO queryOBBacklogDataCKNew(OutboundBacklogQueryBO queryBO) {
    log.info(" queryOBBacklogDataCK入参：{}", JSON.toJSONString(queryBO));
    // 公共条件-卡最近十天时间
    String commonStartTime = DateUtils.getTime(DateUtil.format(new Date(), DateUtil.FORMAT_DATE), DateUtils.ELEVEN_AM, 1, -10);
    String commonEndTime = DateUtils.getTime(DateUtil.format(new Date(), DateUtil.FORMAT_DATE), DateUtils.ELEVEN_AM, 1, 1);
    // ... (asynchronous queries omitted for brevity)
    return processResult(preCrossDockInfoCF, preAssignOrderQtyCF, prePickingInfoCF, preSowInfoCF, preDispatchCF);
}