Why Elasticsearch Struggles on Domestic CPUs: Write, Search & Concurrency Issues Explained

Problem Background

A domestic hardware vendor reported that migrating Elasticsearch from Intel servers to domestic CPUs caused write, retrieval, and concurrency regressions. The author investigates root causes across four layers: hardware, JVM, Elasticsearch itself, and benchmarking methodology.

1. Hardware: Many Cores, Weak Single-Thread, Complex NUMA

Domestic ARM server chips (e.g., Phytium FT2500) have many cores but weak single-thread performance. The FT2500 has 16 NUMA nodes with only 8 cores each. A referenced test (https://openeuler.csdn.net/6a0fe831662f9a54cb764d9e.html) showed that pinning processes to cores doubled QPS and reduced latency from 7.5 ms to 2.2 ms. Elasticsearch is memory-access intensive, so the same principles apply.

2. JVM: Wrong JDK Choice Causes Major Overhead

Tencent Cloud's article (https://cloud.tencent.com/developer/article/1805906) highlights the first major pitfall: using an unverified aarch64 JDK. JDK 17 adds vectorization support, and ZGC becomes NUMA-aware. Aliyun's article (https://developer.aliyun.com/article/1646755) also covers JDK/GC tuning for ARM.

3. Elasticsearch: Version, Thread Pools, Write Patterns

Version gaps: On Kunpeng, a team found their production ES 7.3.0 had no official aarch64 package.

The processors setting forces CPU count and can break write thread limits — dangerous on many-core, special-topology domestic CPUs.

Small-batch "trickle" writes create many small segments, triggering frequent flushes and merges, slowing any platform.

References: Aliyun on Kunpeng openEuler ES7 install (https://developer.aliyun.com/article/975727), Heartsuit blog (https://heartsuit.github.io/2021/05/17/ElasticSearch-Install-on-OpenEuler.html), CSDN write optimization (https://blog.csdn.net/youanyyou/article/details/124743194), Zhihu article (https://zhuanlan.zhihu.com/p/55092525).

4. Benchmarking: Unfair Comparisons

Core count, replica count, refresh_interval, and bulk size must be aligned; otherwise conclusions are meaningless.

Industry Survey: 15 Public Articles → Five Problem Categories

Weak single-core + many cores : Symptom — merge, aggregation, coordinating node bottlenecks. Common fix — reasonable shard splitting, avoid hot shards.

NUMA / interrupts : Symptom — high concurrency causes jitter, throughput non-linear. Common fix — core pinning, multi-instance per NUMA node, soft-irq binding.

JDK / GC : Symptom — high CPU but low throughput. Common fix — use validated aarch64 JDK, compare G1 vs ZGC under load.

Version / package : Symptom — missing packages, plugin incompatibility. Common fix — lock down "OS + JDK + ES + plugins" version matrix.

Operational safety net : Symptom — hard to root-cause faults, no owner. Common fix — engage vendor with domestic deployment experience.

Xinchuang (信创) users report weak post-fault recovery; many faults cannot be root-caused.

ARM itself is not slow: Elastic's official benchmark shows Azure's new-gen ARM VMs achieve 37% higher indexing throughput than previous-gen ARM VMs (https://www.elastic.co/search-labs/blog/azure-arm-elasticsearch-performance). The key is specific chip and tuning, not the "ARM" label.

Proposed Solution: Easysearch

Why "replace the base" instead of "force-tune ES"?

4.1 Compatibility (vendor claim): Supports Huawei, Phytium, Hygon, Loongson, Kunpeng, Zhaoxin, Sunway CPUs; Kylin, UOS, Euler, Deepin OSes.

4.2 Syntax compatibility (vendor claim): ES-compatible DSL, no business code changes; vector search being added (https://mp.weixin.qq.com/s?__biz=MzI2NDY1MTA3OQ==∣=2247503985&idx=1&sn=8848b2aad5c467f7712cf10506a459e1&scene=21#wechat_redirect).

4.3 Third-party validation: Passed CAICT (China Academy of Information and Communications Technology) first batch search-type database capability test, April 2023.

4.4 Clear accountability: Vendor owns performance issues — someone investigates and stands behind the result.

Not suitable when: Business depends on Elastic commercial features or latest versions; client mandates Elastic official subscription; compatibility boundaries unverified.

POC recommendation: Same data, same scripts, target domestic CPU, run ES and Easysearch with shards, replicas, refresh, bulk, concurrency, JDK fully aligned , then compare write, search, concurrency, and P99.

DIY Tuning: 11 Steps, Still No Safety Net

Lock OS, kernel, JDK, ES, plugin version matrix

Compare JDK/GC combinations

Map NUMA topology, decide pinning vs multi-instance

Tune soft-irq, I/O scheduler, huge pages, swap

Calibrate thread pools and processors setting

Redesign shard/index strategy to prevent small-file storms

Re-baseline refresh, translog, merge, bulk

Verify analyzer, plugin, monitoring agent arch compatibility

Establish fair benchmark aligned with Intel baseline

Build fault-location process, run long-stability tests

Repeat from step 1 on every upgrade

Most domestic teams refuse to own ES-on-domestic-CPU because they control neither the ES kernel nor the CPU's low-level behavior. Elastic's China model works through cloud partners (Tencent Cloud 2018, 2026; Alibaba Cloud 2017) — no public evidence Elastic directly supports domestic CPU tuning. Result: every vendor says "not my problem."

Personal Reflections

Clarify goal: Compliance-driven "domestic substitution" targets business SLA, not Intel parity.

Performance gap is likely missing tuning expertise, not CPU deficiency. Ten years of ES experience is x86-centric.

Fair comparison matters: Core count ≠ performance; mismatched parameters invalidate comparisons.

Accountability is real cost: Saved procurement fees become troubleshooting time, downtime risk, and headcount.

Don't oversell Easysearch: Honest framing — if compliance + vendor accountability are core needs, Easysearch deserves a POC; if native ES capabilities are paramount, budget the full DIY tuning cost.

The hardest part of domestic substitution isn't "can it run" — it's "who owns it when it breaks."

References

Tencent Cloud: ES on domestic ARM — first big pit (https://cloud.tencent.com/developer/article/1805906)

Plantegg: Phytium FT2500 performance test (https://plantegg.github.io/2021/05/15/%E9%A3%9E%E8%85%BEARM%E8%8A%AF%E7%89%87%28FT2500%29%E7%9A%84%E6%80%A7%E8%83%BD%E6%B5%8B%E8%AF%95/)

CSDN: Phytium NUMA tuning (https://blog.csdn.net/d1306937299/article/details/128118982)

Aliyun: Extreme ES tuning (https://developer.aliyun.com/article/1646755)

Aliyun: Kunpeng openEuler ES7 install (https://developer.aliyun.com/article/975727)

CSDN: ES write optimization (https://blog.csdn.net/youanyyou/article/details/124743194)

CSDN: Loongson ES deploy (https://blog.csdn.net/qq_41324075/article/details/135523604)

Infoobs: 10 questions on server xinchuang tech roadmap (https://www.infoobs.com/article/20231204/62897.html)

openEuler: Deep dive on xinchuang servers (https://openeuler.csdn.net/6a0fe831662f9a54cb764d9e.html)

Juejin: Elastic official ARM performance blog translation (https://juejin.cn/post/7371809217790001206)

Elastic × Tencent Cloud 2018 (https://www.elastic.co/about/press/tencent-cloud-elasticsearch-service-a-new-partnership-in-china-and-beyond)

Elastic × Tencent Cloud deepened 2026 (https://www.elastic.co/blog/elastic-and-tencent-cloud)

SiliconANGLE: Elastic × Alibaba Cloud 2017 (https://siliconangle.com/?p=410883)

INFINI Easysearch product page (https://infinilabs.cn/products/easysearch/), plus two WeChat articles on financial sector migration and large-enterprise wins.