Understanding Elasticsearch’s Document Retrieval Process

Document Query Steps

Overall query flow

Single Document

Steps to retrieve a document from primary or replica shard:

Client sends get request to Node 1.

Node determines the document’s _id belongs to shard 0 and forwards the request to Node 2 (a replica).

Node 2 returns the document to Node 1, which forwards it to the client.

Coordinator node balances load by round‑robin polling of all replica shards for each request.

If the document is indexed on the primary but not yet replicated, a replica may report “not found” while the primary returns the document. After a successful index request, the document is available on both primary and replicas.

Multiple Documents (mget)

Steps for retrieving multiple documents with a single mget request:

Client sends mget request to Node 1.

Node 1 builds a multi‑document request per shard and forwards them in parallel to the nodes hosting the relevant primary or replica shards. After all replies are received, Node 1 assembles the response and returns it to the client.

Detailed Document Read Process

Search systems typically use a two‑stage query: first find matching DocIDs, then fetch the full documents (query_then_fetch).

In the initial query phase, the request is broadcast to every shard copy. Each shard builds a priority queue of size from + size containing matching document IDs and scores. The shard returns these IDs and scores to the coordinating node, which merges them into a global sorted list.

During the fetch phase, the coordinating node identifies which documents need to be retrieved and issues GET requests to the relevant shards. Each shard loads and enriches the document and returns it. Once all documents are fetched, the coordinating node returns the final result to the client.

Read Mechanism in Elasticsearch

Read consistency means a successful write must be visible to the next read. Search tolerates some delay, while NoSQL databases often require strong consistency.

Lucene Read APIs

public TopDocs search(Query query, int n);
public Document doc(int docID);
public int count(Query query);
... (other APIs)

These APIs provide basic search, document retrieval, and count functions; Elasticsearch builds higher‑level read features on top of them.

Elasticsearch Shard Replication

Each shard has one primary and multiple replicas. Queries can be served by any replica or the primary, improving read throughput. The preference parameter (e.g., _local, _primary, _replica) controls which copy is used. Choosing a replica may return stale data; selecting the primary guarantees the latest version.

Distributed Query Execution

Data is routed to shards based on the _routing value at index time. A search request must query all shards of the target index; each shard returns its top‑N results, which the coordinating node merges using a priority queue to produce the final top‑N list.

Request expansion (one user request becoming one request per shard) can cause high memory usage for deep pagination. Using search_after instead of offset‑based pagination reduces the amount of data each shard must return.

Query Phase Steps

Create Search Context – stores intermediate state (over 50 fields).

Parse Query – converts the query source into Lucene query objects.

Get From Cache – checks if the request can be served from cache.

Add Collectors – builds a list of collectors (FilteredCollector, PluginInMultiCollector, MinimumScoreCollector, EarlyTerminatingCollector, CancellableCollector, EarlyTerminatingSortingCollector, TopDocsCollector, etc.) to gather and filter results.

lucene::search – invokes Lucene’s IndexSearcher.search on each segment, computes scores, and performs first‑phase scoring.

Rescore – optional second‑phase scoring if the request includes a rescore configuration.

suggest::execute – executes suggestion requests if present.

aggregation::execute – runs aggregation logic using collectors.

Fetch Phase

After the top‑N DocIDs are identified, the fetch phase retrieves full document content (stored fields, source, doc values, scripts, highlights, etc.) from the shards. Elasticsearch registers eight default fetch sub‑phases (ExplainFetchSubPhase, DocValueFieldsFetchSubPhase, ScriptFieldsFetchSubPhase, FetchSourceSubPhase, VersionFetchSubPhase, MatchedQueriesFetchSubPhase, HighlightPhase, ParentFieldSubFetchPhase) and allows custom plugins.

The fetch phase runs only for the final top‑N documents, reducing I/O and CPU compared to fetching every candidate document during the query phase.