Mastering Elasticsearch Query Limits: Tips to Boost Performance

Result Size Limit

Default limit

Elasticsearch limits the number of documents returned per search query. The size parameter defaults to 10 . Setting a high size increases memory usage, CPU load, and disk I/O.

Maximum result window

The default maximum result window is 10,000 documents. Exceeding this value triggers an illegal_argument_exception (e.g., "Result window is too large, from + size must be less than or equal to: [10000] but was [100000]"). The limit is controlled by the index.max_result_window setting.

Solutions for deep pagination

search_after – pagination without from and size, offering better performance.

GET /my-index/_search
{
  "size": 1000,
  "query": { "match": { "title": "Elasticsearch" } },
  "sort": [ { "timestamp": "asc" }, { "_id": "asc" } ],
  "search_after": [1609459200000, "my_document_id"]
}

Scroll API – suitable for extracting large data sets in a single operation; keeps a server‑side snapshot for consistent traversal.

GET /my-index/_search?scroll=1m
{
  "size": 1000,
  "query": { "match": { "title": "Elasticsearch" } }
}

Note: Scroll is not intended for real‑time user requests because it consumes server resources.

Max Clause Count

Background

Lucene, the core engine of Elasticsearch, protects against excessively large Boolean queries that could exhaust memory and CPU.

Default limits

Earlier versions capped max_clause_count at 1024 . Starting with Elasticsearch 7.0 the default increased to 4096 . Exceeding the limit aborts the query with an exception.

Changes in 8.0+

In Elasticsearch 8.0.0 the setting indices.query.bool.max_clause_count was deprecated and no longer influences query behavior.

Dynamic adjustment

Elasticsearch now adjusts the maximum clause count based on the search thread‑pool size and JVM heap memory. The minimum is 1024; on a node with 30 GB RAM and 48 CPUs the limit can reach roughly 27 000.

Recommendations

Avoid overly complex Boolean queries; design queries conservatively.

Field Data Limits

Problem

Aggregations and sorting load field data into heap memory; high‑cardinality fields can cause OutOfMemoryError.

Optimization strategies

Limit field‑data cache size – set indices.fielddata.cache.size (e.g., 40%) to cap heap usage. indices.fielddata.cache.size: 40% Use pre‑aggregated metrics – store frequently aggregated values as separate fields during indexing to reduce real‑time computation.

Avoid aggregating on text fields – use keyword fields instead, as text fields require analysis before aggregation.

Join Operations Limits

Problem

Complex nested, parent‑child, or join‑style queries increase cluster load and may hit max_clause_count, causing query rejection.

Recommendations

Flatten data model – store related data in a single document to avoid joins, accepting some redundancy.

Perform joins in the application layer – retrieve data separately and combine it in the application rather than relying on Elasticsearch joins.

Query Throughput Limits

Understanding throughput

Each Elasticsearch cluster has a maximum query throughput determined by node count, hardware specs, and query complexity. Exceeding capacity leads to higher latency or request rejection.

Example calculation

Assume each query takes 1 second, the cluster has 3 data nodes, and each node provides 13 search threads.

Maximum throughput = nodes × threads per node / query time
Maximum throughput = 3 × 13 / 1 = 39 queries/second

Protection mechanisms

The search queue returns HTTP 429 when the queue is full.

A circuit‑breaker aborts operations that exceed memory or resource thresholds.

Optimization suggestions

Monitor query latency and resource consumption with Kibana or Elastic Stack Monitoring.

Identify slow queries and rewrite them or add appropriate indexes.

Scale hardware: add data nodes or upgrade CPU, memory, and storage.

Set sensible request timeouts to avoid long‑running operations.

Implement exponential backoff when receiving HTTP 429 before retrying.

Conclusion

Understanding Elasticsearch's query limits is essential for performance tuning and cluster stability. Configure parameters wisely, monitor resource usage, and design efficient data models to avoid hitting these limits and to deliver a faster, more reliable search experience.