Why Does Elasticsearch Refresh Take 1‑5 Seconds? A Deep Dive into Index Settings and Soft Delete

Background

After writing data, the refresh operation sometimes takes 1–5 seconds, which is far longer than expected. A series of tests were created to identify factors that influence refresh latency, including write type (insert vs. update), the proportion of deleted documents, and index‑level settings such as forcemerge and soft‑delete.

Test environment

Old index : 24 primary shards, 1 replica, long‑term forcemerge, max segment 33 GB, ~8 % deleted documents.

noforcemerge index : 24 primary shards, 1 replica, reindexed, max segment 5 GB, 0 % deleted.

nosoftdelete index : soft‑delete disabled, 24 primary shards, 0 replica, reindexed, max segment 5 GB, 0 % deleted.

Test methodology

All indices were written at a constant rate of 2 k documents per second. For each test window the following metrics were recorded:

Write speed (2 k/s)

Growth of the deleted ‑document ratio refresh latency (both normal and refresh_external)

CPU utilization

Key measurements (summarised):

---------------------------------------------------------------
Index / Scenario          | Refresh latency | Deleted % | CPU
---------------------------------------------------------------
Old index – update          | 3 s – 12 s      | 8 % – 32 %| 50‑100 %
Old index – insert          | 200 ms – 400 ms | ~22 %    | 50‑100 %
Old index – forcemerge add  | 300 ms – 800 ms | 0‑2 %    | 50‑100 %
Noforcemerge – update       | 1 s – 3 s       | 0‑9 %    | 50‑100 %
Noforcemerge – low‑update   | 500 ms – 1.5 s  | 7‑6 %    | 30‑60 %
Nosoftdelete – update       | 200 ms – 300 ms | 0‑8 %    | 10‑40 %
---------------------------------------------------------------

Observations

Pure update operations cause noticeably higher refresh latency compared with pure inserts.

Reducing the proportion of deleted documents also lowers the high refresh latency.

In the noforcemerge index, update tests showed fewer refresh occurrences (still 100 % of the time) while merge‑thread activity increased significantly.

Disabling soft delete ( nosoftdelete) dramatically reduces refresh latency, and the latency no longer correlates with the deleted‑document ratio.

Conclusions

Large‑segment strategy

The large segment strategy does not add extra resource consumption for daily queries or writes, and it achieves the expected automatic cleanup of deleted documents.

Factors affecting refresh latency

Soft delete : Enabling soft delete directly increases refresh latency; disabling it reduces latency but is generally not recommended because the setting is immutable after index creation and is slated for removal in future Elasticsearch versions.

Write operation type : Updates (especially when soft delete is enabled) significantly increase refresh time, while pure inserts have little impact.

Deleted‑document proportion : A higher ratio of deleted documents leads to longer refresh durations.

Soft delete – technical details

Soft delete is used for shard‑level data synchronization and recovery and is part of Elasticsearch’s distributed fundamentals. It is enabled by default, can only be set at index creation, and cannot be toggled later. Future Elasticsearch releases plan to remove this parameter, which is why modifying soft_delete is discouraged.

References

https://www.elastic.co/guide/en/elasticsearch/reference/7.10/index-modules-history-retention.html

https://mp.weixin.qq.com/s/_l8JAtqK_NOSP8b7OqSVDg

Illustrations